The bodies were found in the Tal al-Hawa and Al-Sinaa districts, the agency said. Israeli troops had pulled out after days of battles with Hamas militants, the agency and residents said. This was not immediately confirmed by Israel.
Civil defence and medical teams had "recovered approximately 60 martyrs since the moment the Israeli occupation army withdrew this morning," an agency statement said.
"There are still missing people under the rubble of destroyed homes, which is difficult for our crews to reach," agency spokesman Mahmud Bassal said in the statement. "There are reports that many people are missing since the first day of the incursion."
Israeli forces moved into Tal al-Hawa and neighbouring districts including Al-Rimal again this week after ordering civilians to evacuate on Monday. Residents have reported widespread new destruction.
Israel has since warned residents of virtually all of Gaza City that it is now a "dangerous" combat zone.
"There are many calls for help but we just cannot reach them," Bassal said. "We just do not have enough crews."
He added that the Sabha Medical Centre, near the Gaza City district of Shujaiya, which provides care for 60,000 residents, had been destroyed in the new fighting.
Shujaiya was the scene of two weeks of battles between Israeli and Hamas forces, some of the fiercest since the war started with the Hamas attacks on Israel on October 7.
The civil defence agency said Thursday that 60 bodies had been found in Shujaiya after Israel's withdrawal.
The health ministry said 32 bodies had been taken to different hospitals across the territory following night-time Israeli air strikes and shelling.