"Stay strong you who are still there," cried 17-year-old former hostage Agam Goldstein with tears in her eyes.
The teenager was freed in a deal with Hamas at the end of November.
About 250 Israelis and foreigners were taken hostage by militants on October 7.
The army says 129 are still being held in Gaza, including 34 who are presumed dead.
In another moment of high emotion at the rally, Ofri Bibas appealed for the lives of her brother Yarden, his wife Shiri and their sons, Ariel, 4, and one-year-old Kfir -- the youngest of the hostages.
After a massive anti-government demonstration in Tel Aviv on Saturday, which also heard emotional calls to free the hostages, organisers of the Jerusalem rally attempted to be apolitical.
But Lishay Meran, whose husband Omri is among those held captive in Gaza, took aim at the country's Prime Minister Benjamin Netanyahu who is frequently accused of not doing enough to free them.
"We were abandoned on October 7, and we have been abandoned since," said the mother of two as protesters carried banners reading "Free them now!" and "Bring them home".
Other hostage families have accused Netanyahu of trying to blacken their reputations and accusing them of being "traitors" for protesting in wartime.
But Netanyahu vowed on Sunday that the war would not end until all the hostages are released.
The Gaza war broke out on October 7 with an attack by Hamas militants that resulted in the deaths of 1,170 people, mostly civilians, Israeli figures show.
Israel's retaliatory offensive has killed at least 33,175 people in Gaza, mostly women and children, according to the health ministry in the Hamas-run territory.
The latest round of indirect talks between Israel and Hamas were set to resume in Cairo on Sunday, with the United States, Egypt and Qatar as mediators.
Israel and Hamas have accused each other of dragging out the negotiations.
