Fourth seed Zverev, who had lost in the semi-finals at Roland Garros in each of the past three years, fought back to beat Ruud 2-6, 6-2, 6-4, 6-2 in a repeat of their last-four clash 12 months ago.
Zverev won just seven games when the pair met in the 2023 semis and a similar story was on the cards after Ruud cruised through the opening set.
But the German raised his game, reaching his first French Open final and keeping his hopes of a maiden Grand Slam title alive.
Zverev holds a career 5-4 winning record against Alcaraz, including a victory in their only previous Roland Garros meeting in the 2022 quarter-finals.
Alcaraz earlier battled past incoming world number one Jannik Sinner in five sets.
There will be two first-time finalists for the first year since Rafael Nadal defeated Mariano Puerta for the first of his 14 titles in 2005.
The 27-year-old Zverev's only previous major final appearance saw him blow a two-set lead to lose to Dominic Thiem at the 2020 US Open.
"I was not ready then to win my first Grand Slam final," said Zverev, the first German man to make the final since Michael Stich in 1996.
"I'm definitely not a kid any more, I'm a little older. If not now, then when?"
A German court dropped a case against Zverev over allegations he assaulted an ex-girlfriend earlier on Friday, after a settlement was agreed.
Ruud was targeting a fourth Grand Slam final, but the two-time Roland Garros runner-up was well below his best and seemed to struggling with illness.
The Norwegian seventh seed had not played since Monday after enjoying a walkover in the quarter-finals due to Novak Djokovic's withdrawal from the tournament with a knee injury.
Zverev had spent more than eight hours on court during two five-setters in the third and fourth rounds, before beating Alex de Minaur in the last eight.
Ruud looked the fresher of the two as he raced into a one-set lead.
But Zverev was much improved in the second set, breaking twice while hitting 15 winners against just four unforced errors to level the match.
A break in the fifth game of the third set put him in control against a struggling Ruud.
He served it out for a 2-1 edge in sets, despite briefly faltering with successive double-faults and having to save a break point.
Ruud took a trip to the medical room after the third set.
But he was broken again in the first game of the fourth before Zverev moved to the brink of the final by giving himself breathing space with another break for a 5-2 lead.
He showed no signs of nerves, confidently serving it out at the first opportunity with an ace.
