Mumbai firefighters were also trying to extinguish burning debris that could further endanger people who were still trapped in the building in Bhendi Bazaar, an area crowded with markets and shops.
"The fire in the debris continues to cause a hindrance in the rescue work and is possibly suffocating victims trapped in confined spaces," said Mumbai Fire Brigade Chief P.S. Rahangdale.
Thirty-seven people were rescued, many of whom were hospitalized. Seventeen were being treated and the remainder had been released, according to a Mumbai police spokesperson.
A spokesman for the Municipal Corporation of Greater Mumbai said the building was 117 years old and had been deemed unsafe years ago, raising questions over why it was still inhabited.
"The building was served an 'unsafe to live' notice in 2013," said the spokesman, Vijay Khabale-Patil.
"Ten families used to live in the building and some of them had vacated the property recently. So we don't have an actual count of the missing people."
An official told CNN there are likely to be hundreds of other buildings in the city in a similar condition.
It was a chaotic scene as the rescue operation took place, with an excavator digging through the mountain of rubble, dumping broken concrete, shards of thin wood and bits of twisted metal into trucks. Whistles sounded as ambulances drove up to the site in expectation of more bodies.
As night fell, the sound of megaphones and desperate shouts continued as the call to prayer sounded from the many mosques in the area.
The building collapse comes after heavy rains and flooding in Mumbai this week, which have killed at least five people. Authorities would not say if the collapse was connected to the rainfall.
South Asia has seen a historic amount of rain and flooding this month, the height of monsoon season.
More than 1,200 people have been killed in India and Bangladesh and some 41 million have been affected by flooding, according to the International Federation of Red Cross and Red Crescent Societies. Nepal has also been hit hard.
Thursday's building collapse is the second significant one to hit India's financial hub this summer. More than a dozen people were killed when a five-story building collapsed in the Ghatkopar neighborhood in July
In May, a wall collapsed at a wedding, killing 24 people.
Deadly infrastructure accidents are not unusual in India. Critics say construction projects often lack proper oversight and safety controls.