(CNN) -- Ukrainian President Viktor Yanukovich pledged Thursday to make his country "a bridge between East and West" after being sworn into office in Kiev.
Yanukovich, who was ousted from office in 2004 in the pro-West "Orange Revolution," beat Prime Minister Yulia Tymoshenko in a bitterly fought election earlier this month that has left the country deeply divided.
Yanukovich enjoys support among the Russian-speaking people living in eastern Ukraine, while Tymoshenko is mainly supported by the Ukrainian speakers in the western part of the country. There is concern Yanukovich will move the former Soviet state more closely into Moscow's orbit and many fear further political turmoil.
But in his inaugural address, Yanukovich said it was in Ukraine's interest to build close relations with Russia, the European Union and the U.S.
"Being a bridge between East and West, an integral part of Europe and the former Soviet Union at the same time, Ukraine will choose a foreign policy that will allow our country to get the most out of the development of equal and mutually beneficial relations with Russia, the European Union, the United States and other countries that influence development in the world," he said.
Yanukovich also said he would put Ukraine on a "path of accelerated development" to save Ukraine from socio-economic collapse and called for cooperation between president, parliament and government.
In one of his first action as presidents, Yanukovich said he had reduced the costs of maintaining a presidential administration by 20 percent. The money saved would go to the fifth of Ukrainians living in poverty, according to a statement from his office.
Yanukovich was declared the winner of the presidential election in 2004, but the legitimacy of the vote was questioned and he faced accusations of stealing the race.
Yanukovich's win was annulled following the Orange Revolution uprising, with former President Victor Yushchenko winning the revote.
Tymoshenko stood alongside Yushchenko during the Orange Revolution, but later political infighting broke down their alliance. In a recent interview with CNN, Yanukovich attributed his dramatic comeback to the current economic situation, saying people voted for him because they believe he can improve the economy.
"I want to tell the people of Ukraine ... we will repay the arrears of wages and pensions, what the current government never managed to do," he said Thursday. "Given the dire situation in public finances, we will do this by reducing the costs of (the) bureaucratic system. We will start with ourselves."