Authorities said at least 24 people died after a section of the under-construction bridge came crashing to the streets of Girish Park around midday Thursday.
The collapse of about 100 meters (about 328 feet) of the Vivekananda Overpass buried moving cars and pedestrians in the crowded neighborhood north of the city's center.
Eighty people were rescued from the rubble, Major General Anurag Gupta from the National Disaster Management told CNN.
At first, while the dust hung in the air, people desperately dug into the rubble with shovels and even their hands. Later, an army of hard-hatted rescue crews arrived. Bright lights illuminated the scene as heavy machinery lifted wide slabs of concrete that fell onto the streets.
Nobody knows how long the rescue effort will last or how many people may have died, Gupta said.
"We cannot predict how many people are still under wreckage," he said. "We can't tell since it's a bridge and not a building."
At least 75 people suffered injuries in the collapse, Gupta said. "We believe very few people are still inside the rubble," he told CNN.
At one point, Kolkata police Sgt. Saurav Benerjee said more than 100 were unaccounted for, though that doesn't mean they were all necessarily trapped.
Frantic search for family
Shabana Farooqui, 35, was on her way to pick up her two children from school when the overpass fell, her family told CNN IBN. They said they visited hospitals and the morgue and don't know if she's dead or alive.
They are not alone.
Video showed swarms of people on the scene in Girish Park, looking feverishly for family members and trying to help clear away the rubble.
Some carried out pieces of debris, and others used fire hoses to spray water around it.
Authorities evacuated several nearby apartment buildings over fear of structural damage and worries that other sections of the 2-kilometer overpass might fall, CNN IBN said.
Company executive calls it 'God's act'
"It's nothing but a god's act. So far in 27 years, we have constructed several number of bridges ... it never happened," K Panduranga Rao of the Hyderabad-based company told reporters.
But overpasses have collapsed in the city before. In March 2013, a fully constructed overpass collapsed in eastern Kolkata. A truck fell into a canal, but the people inside were rescued. It's not known what company built that overpass.
The Vivekananda Overpass project had been plagued by delays and was supposed to have been finished five years ago.
CNN's Mallika Kapur in Mumbai said that's not unusual in India. Several long overpasses that extend more than a mile have been built in Kolkata in the past decade.
The construction company official's comments drew sharp criticism on social media, as some argued that the engineers and executives behind the project should be punished.
Other overpass accidents
The company website said the Vivekananda Overpass project was a composite steel structure, full of utilities, and was being built in a congested area that hampered movement of heavy equipment. It isn't the first time there's been a deadly road construction accident in India in recent years.
India has experienced other overpass collapses over the years.
In Bhagalpur in 2006, an overbridge fell on an express train, killing 37 passengers.
In 2009, scaffolding on a bridge under construction collapsed in Kota, killing 30 workers.
In 2014, three laborers were killed when a portion of an overpass under construction collapsed in the Parle Point area of Surat, a Surat official told the PTI news agency.
Prime minister 'shocked and saddened'
"My thoughts and prayers are with the families of those who lost their lives in Kolkata," Modi added. "May the injured recover at the earliest."
Speaking on local TV from the collapse site, Mamata Banerjee -- chief minister of West Bengal state, where Kolkata is the capital -- announced that relatives of those killed in the collapse would get roughly the equivalent of $7,500.
Those critically injured, meanwhile, would get about $3,000 each.
CNN's Sumnima Udas reported from New Delhi, and CNN's Joshua Berlinger, Ralph Ellis and Greg Botelho reported and wrote from Atlanta. CNN's Sugam Pokharel, Yazhou Sun, Mohammed Tawfeeq and Catherine E. Shoichet contributed to this report.