The development came at what the Justice and Development Party, or AKP, called an "extraordinary congress" in Ankara.
Members chose Binali Yildirim, an AKP veteran and the country's transport minister, according to Turkey's semiofficial Anadolu Agency.
The move paves the way for Yildirim to take over as the next Prime Minister.
"I promise that I will never drop this sacred flag you have given me as new chairman," Yildirim was quoted as saying after the congress.
Davutoglu stepping down as Prime Minister
Yildirim will take the place of Ahmet Davutoglu, who said earlier this month he will resign as Prime Minister.
Davutoglu's announcement came after several differences with Erdogan.
Erdogan, who was Prime Minister from 2003 to 2014, has been vocal about his ambitions to change the country's constitutional arrangements to move to a presidential system.
The prime minister, by law, holds more power than the presidency. Critics see Erdogan's vision for the office as part of his authoritarian instinct for more power.
Erdogan's supporters have criticized Davutoglu for not doing enough to push for the presidential system, although Davutoglu is on record as saying he supports such a change.
Yildirim on Sunday stressed the need for a presidential system.
"The country needs a new constitution, and in this new constitution, it is the presidential regime which needs to be the priority," Yildirim said.
Other issues on which Erdogan and Davutoglu were seen to differ included Kurdish militants -- whom Davutoglu had said that he may be willing to sit down with again -- and the jailing of journalists and academics, a trend much criticized by Erdogan's critics.
The largely symbolic presidency
Erdogan handpicked Davutoglu, a former academic and diplomat, to succeed him as Prime Minister in 2014 when he was elected to the largely symbolic presidency.
Davutoglu's decision to step down was done in an amicable fashion. He attended the congress in Ankara, expressed his loyalty to the AKP and congratulated his successor.
And he will continue in his role as an AKP parliamentarian.
"AKP has never been and will never be the party of one faction, of one clan. The AKP is the party of the Turkish people in its totality and thus will remain," he said. "Now, I congratulate our candidate and Transport Minister Binali Yildirim and wish him success."
Erdogan accepted Davutoglu's resignation and asked Yildirim to form the country's new government. The President asked Davutoglu and other Cabinet members to remain in their posts until the formation of the new government.
ISIS, Kurdish militancy, refugees
Turkey, a NATO member with a powerful economy, is facing growing instability amid escalating problems with ISIS, Kurdish militants and an influx of refugees from neighboring Syria. The government says the refugee crisis has cost it more than $10 billion so far.
At the congress, Yildirim called for "uninterrupted" operations against the Kurdistan Workers' Party fighters until the group ends its "armed actions." Turkey has been fighting the PKK for years.
Turkey struck a deal with the European Union in March under which people who cross into Europe illegally are being sent back to Turkey. For every Syrian sent back to Turkey under the plan, a vetted Syrian refugee will go from Turkey to Europe to be resettled.
Yildirim raised the issue of Turkey's future with the European Union. The country has sought membership in the group, but there have been roadblocks.
"My brothers, there is one thing that needs to be done by the European Union. This confusion over Turkey's full membership and the migrant issue has to be brought to an end. The time has come for the EU to eventually reveal what it thinks about Turkey," Yildirim said.