(CNN) -- Retired Gen. Colin Powell has a choice blend of political and military experience, and many thought he'd make a great president, but Powell said Wednesday that he just didn't have it in him.
Colin Powell, left, embraces Nobel laureate Elie Wiesel at the inauguration Tuesday.
His wife, Alma, had feared that such an endeavor would change their family life. She also had concerns about Powell's safety, he said Wednesday on CNN's "American Morning."
"But I was a soldier. That wasn't my concern," he said. "I never found inside of me the internal passion that you've got to have to run for elected office."
Many GOP pundits had hoped Powell, who served as secretary of state under President George W. Bush and chairman of the Joint Chiefs of Staff under Bush's father and President Clinton, would make a run at the Oval Office.
The former four-star general and Vietnam War veteran's military credentials are staunch, rivaled only by his accomplishments as a statesman. Watch Powell explain why the White House wasn't for him »
In addition to chairing the Joint Chiefs, the lifelong Republican served as national security adviser under President Reagan and oversaw the 1989 invasion of Panama that toppled Gen. Manuel Noriega as well as Operation Desert Storm during the 1991 Gulf War. He has two Presidential Medals of Freedom to his name.
As Bush's secretary of state, he spearheaded efforts to increase U.S. foreign assistance throughout the world, and he helped develop Bush's HIV/AIDS prevention and treatment program, which is credited with being the largest such endeavor ever.
However, Powell drew heavy criticism over his remarks before the United Nations that Iraq had weapons of mass destruction. Powell made the assertion while attempting to convince the world body that the U.S. should go to war with Iraq.
Powell abruptly retired from Bush's Cabinet in 2005, but he did not cite his time a secretary of state Wednesday when explaining why he never sought the presidency.
"I never woke up a single morning to go think about this, to talk to people about it and find in my heart and soul the passion that a Barack Obama or a John McCain or a George Bush or a Bill Clinton had," he said. "It just wasn't me, and you've got to be true to yourself, and I've tried to be true to myself."
The stance is not new for Powell, who held a news conference in November 1995 to announce that he would not run for president.
"To offer myself as a candidate for president requires a commitment and a passion to run the race and to succeed in the quest," he said then, "a passion and commitment that, despite my every effort, I do not yet have for political life, because such a life requires a calling that I do not yet hear."
No stranger to advising presidents, Powell told CNN that Obama has a great deal of work ahead. Not only does he have to deal with the crisis du jour -- like righting the economy at home -- he needs to keep his eyes on crises abroad while never foregoing long-range planning. Watch what Powell says Obama must do »
"You can never avoid it," he said. "Once you're the president, every crisis comes to your desk, and you have to deal with it. ... But that doesn't mean that you can't stand back and look farther out."
Repairing America's broken image and poverty alleviation -- both at home and abroad -- will pose major challenges, he said, and both have wide-ranging implications.
Poverty incubates terror, and it is incumbent upon the U.S. to stamp out the hunger, unemployment and illiteracy that breed suicide bombers and other insurgents, Powell said. He called poverty alleviation "one of the most important challenges facing the world."
"For those of us who are wealthy, we should reach out not only to our own citizens who need help, but the rest of the world," he said. "If you want to get rid of sources of terrorism, if you want to get people moving in the right direction, you've got to help them get jobs, clean water, health care for their kids, educate their kids -- and America has a great responsibility to do this."
Obama also needs to consider what other nations think of the U.S. and help repair the American reputation. The challenge is daunting, but Powell said America's good standing is "recoverable."
The sea of diverse faces in attendance at Obama's inauguration ceremony speaks to the reverence that America still enjoys in the world, said the son of Jamaican immigrants and father of three.
If Obama can turn the economy around and rectify situations in Iraq and other parts of the Middle East, he will begin repairing the nation's image, "because there's still a solid, residual level of affection and respect for the United States of America."
But Powell acknowledges that his former boss and Obama's predecessor made things difficult.
"Iraq and the Guantanamo situation -- which I have been preaching for years should have been closed years ago -- Abu Ghraib and a lack of progress in the Middle East and a certain way in which we deal with the world on international issues, I think damaged our reputation with the world," he said.
Obama is off to a good start, he said, and is doing the right thing by reaching across the aisle for help. It was gracious of Obama to hold a dinner in Sen. John McCain's honor after a testy campaign and gracious of McCain to accept, and Powell said he hopes that spirit will permeate Obama's presidency.
Powell conceded, though, that bipartisanship won't always be the answer to Obama's problems and said the 44th president will need to rely on the wisdom of those who forged the country more than two centuries ago.
"We're supposed to be a partisan country. That's the way the founding fathers intended for us to move forward," he said.
"People have strong views on both sides of an issue. Argue it out. Fight it out, just like they did in that summer of 1787 when they were writing the Constitution -- and then ultimately both sides make compromises in order to achieve consensus, and then you move the country forward."
CNN's John Roberts contributed to this report.