The Philippine coast guard said 140 people survived.
The MB Nirvana tipped over Thursday, roughly 200 meters (220 yards) from shore only minutes after it set off from Ormoc City, in Leyte province. It was bound for the town of Pilar on Camotes Island, in Cebu province.
Three Americans are among the survivors. They are Rhome and Chip Nuttall, and Larry Drake, according to Philippine Red Cross Chairman Richard Gordon.
Rhome Nuttall, who helps run a medical mission in the Philippines, told CNN that the boat was "very crowded."
"There was a little wave, which is normal, but yesterday that caused the boat to flip to the right," she said Friday. "I ran to the left hoping we could help to balance the boat. When we flipped, I said: 'OK, this is it. OK, this is it.'"
Water came in and washed her out of the boat. Nuttall said she couldn't find her husband for about five minutes, and that everyone panicked.
She remembered hearing a loud noise before the vessel capsized.
"I think it was that big thud. I think that was cargo that shifted and that caused the boat to flip," Nuttall said.
Cause of disaster under investigation
Investigators will examine what caused the boat to overturn.
A CNN Philippines reporter quoted the coast guard as saying the boat had left port too quickly and people stood up, throwing the boat off balance. Gordon said he understood the boat overturned because of high winds.
The coast guard is leading the search effort for those still unaccounted for, with divers from the Red Cross and the military also involved.
Poor weather conditions hampered the initial rescue effort on Thursday.
Video footage from the scene showed people of all ages, including a small child, being brought ashore in small inflatable craft. Some survivors were laid on the concrete waterfront, while others were wheeled away on stretchers.
Passengers described as 'common folks'
Gwendolyn Pang, secretary-general of the Philippine Red Cross, described the vessel as a commercial "pump boat" or banka, which is essentially a canoe-style craft with outriggers powered by a small engine.
They're widely used across the Philippines for transporting people and goods as well as for fishing.
Gordon said the boat operates three times a day on the route from Ormoc to Camotes Island.
"The passengers would have been farmers or fishermen, or ordinary businessmen -- common folks," he said.
Rhome Nuttall told CNN that one of the other boats that work the route was broken Thursday, so the MB Nirvana was extra crowded.
"They are not very well off, otherwise they would be on better vessels or take the plane," Gordon said. "But obviously, these are poor folks, simple folks who are trying to eke out an existence."
This style of outrigger boat has no cabins, he said, which should make it easier for divers to find anyone trapped under the canopy or keel of the vessel.