More than 200 people were rescued from damaged structures, many from a 17-story residential building that collapsed in Tainan, officials told CNN. Two bodies were also recovered from the building, officials said.
Taiwan's official Central News Agency reported earlier that a 10-day-old girl and a 40-year-old man had been killed in Tainan.
A man who was staying in a three-story apartment building in Tainan told Chinese state media CCTV the building was shaking violently when the quake hit before sunrise. His room tilted.
The man said the 45-degree tilt resulted in items inside his room spilling all over the floor. Unable to use the stairs, he managed to escape from the window of the second floor.
One woman told CNN affiliate EBC that rescuers had to cut a hole in order to help her family get out.
"Fortunately we were stuck under a space created by a baby crib and a closet door, so that things won't fall on us and air was able to get in," she said from the hospital, where she was receiving treatment for a leg injury. "I was so afraid."
Video shot by CNN affiliate SETTV showed rescuers helping several people from the rubble, including one man who had blood on his face. In another shot, rescuers carefully lower an elderly woman strapped to a backboard.
SETTV's aerial footage showed the collapsed 17-story building, with white smoke or dust still billowing from the destruction.
More than 1,500 people are involved in rescue efforts, the disaster center said. Seven other buildings were damaged.
Taiwan's outgoing President, Ma Ying-jeou, is on his way to the city, EBC and other Taiwanese media reported.
About 1.9 million people live in Tainan,Taiwan's oldest city.
As of 5:30 a.m. (4:30 p.m. ET Friday), 121,672 homes and businesses were without power in Tainan
The tremor was centered about 48 kilometers (30 miles) east-southeast of Tainan, the U.S. Geological Survey said.