More than 350 people were rescued from damaged structures, many from a 16-story residential building that collapsed in Tainan, officials told CNN. At least 16 of the dead were from the Weiguan Jinlong building, CNA said.
As of Saturday night, dozens of children were among at least 153 people still unaccounted for in Tainan, CNA reported, citing the local fire department. A cold wave moving into the area added to their sense of urgency.
CNA reported earlier that a 10-day-old girl and a 40-year-old man had been killed in Tainan.
More than 60 people in the city remain hospitalized, disaster officials said.
Seven other buildings in Tainan were damaged.
"This was strong enough to not only be felt here in the [Taiwanese] capital city of Taipei but also in the southern provinces of China," Elise Hu, an NPR correspondent who was in Taipei when the quake hit, said in an interview with CNN. "Taiwan is very used to earthquakes and tremors, but this is far more significant than the island has seen in quite a while."
Pop star Madonna told concertgoers in Taipei that she felt the tremor in her 17th floor hotel room and it was scary, CNA reported. At the Saturday night show she offered her condolences and asked the audience to keep the victims in their thoughts.
More than 1,500 people are involved in rescue efforts, the disaster center said.
A man who was staying in a three-story apartment building in Tainan told Chinese state media CCTV the building was shaking violently when the quake hit before sunrise. His room tilted 45 degrees, and items inside his room spilled all over the floor.
Unable to use the stairs, he managed to escape from the window of the second floor.
The Weiguan Jinlong building now looks like an accordion from above.
SETTV's aerial footage showed the collapsed building, with white smoke or dust still billowing from the destruction.
"The building essentially collapsed onto itself," Hu said. "When you see the aerial images around Tainan, the rest of the buildings are standing. But this particular apartment complex is as damaged as it is."
One woman told CNN affiliate EBC that rescuers had to cut a hole in order to help her family get out.
"Fortunately we were stuck under a space created by a baby crib and a closet door, so that things won't fall on us and air was able to get in," she said from the hospital, where she was receiving treatment for a leg injury. "I was so afraid."
Video shot by CNN affiliate SETTV
showed rescuers helping several people from the rubble, including one man with blood on his face. In another shot, rescuers carefully lower an elderly woman strapped to a backboard.
Officials, including the country's interior minister, say they will conduct an investigation into the building's collapse, according to CNA.
After residents raised concerns about the safety of the building, Tainan Mayor Lai Ching-te said he will order a probe as well.
Taiwan's outgoing President, Ma Ying-jeou, is on his way to the city, EBC reported.
'Imagine something like this happening during Thanksgiving'
The quake struck as many in Taiwan prepare to celebrate the Lunar New Year.
It's one of the country's biggest holidays, and some people have as many as nine days off, Hu said.
"If you can imagine something like this happening during Thanksgiving holiday weekend or Christmas travel, that's the equivalent of what's happening here in Taiwan right now," she said.
As of Saturday morning (Friday evening ET), about 900,000 homes were experiencing power outages across the country and some 400,000 were dealing with water outages, according to the National Fire Agency.
About 1.9 million people live in Tainan, Taiwan's oldest city.