The duo seized control in the final stages of the gruelling 200-lap race at Saint-Quentin-en-Yvelines velodrome to finish eight points ahead of Italy, with Denmark taking the bronze.
Avoiding a series of accidents, they won all four of the last sprints in a sensational finish to overtake the Italians at the death.
"This all seems a dream," said Oliveira. "I've never won a single race in my life. There have been so many disappointments through the years.
"If someone had said you'd win your first race at the Olympics in Paris, I'd say 'you're kidding me'.
"We're not even close to being the favourites but we ride our hearts out," he added.
Not only was it Portugal's first track title, but their maiden gold of the Paris Olympics.
Notoriously chaotic and unpredictable, the madison is a two-person tactical relay event, featuring a mass start.
Points are accumulated from sprints every 10 laps -- with the top four teams rewarded -- while the final lap offers double points.
Crucially, an additional 20 points can be earned if a team laps the field.
Austria made its tactics clear at the start, going out hard to take the first sprint, catch the peloton and earn an extra 20 points.
But the exertion took its toll and they were later lapped themselves.
Fatigued
Italy and Denmark both gambled with the same move mid-way through the race, hauling in the field with sustained speed.
At the halfway mark, the Italians were eight points clear of the Danes, with Spain and New Zealand joint third.
But no-one had banked on Portugal, who made their move late as others tired, winning three sprints, banking an extra 20 points, the storming home to win the final lap.
"It's incredible, it's such a long race. I cannot lie I was really fatigued after the omnium," said Leitao, who powered to silver in that race on Thursday.
"With 160 (laps) to go, I felt really empty and I told Rui we need to wait because I wasn't sure I could recover.
"But in the end I knew we could do it. We trained to do the last 20 laps or so full gas. We were really focused on doing that."
In other track action, British world champion Emma Finucane and New Zealand's keirin gold medallist Ellesse Andrews both sped into the women's sprint semi-finals.
Germany's Lea Friedrich, who set a new world record in qualifying, also made the grade along with the Netherlands' Hetty van de Wouw.
But Canadian defending champion Kelsey Mitchell crashed out.
Powerhouse Dutchman Harrie Lavreysen was in sizzling form to easily win his men's keirin heat as he zeroes in on a third gold of the Games.
The 27-year-old, who dominated the sprint and team sprint, is gunning to better his bronze from Tokyo.
Australia's Matthew Richardson, who came second to Lavreysen in the sprint final, also progressed, as did Colombian world champion Kevin Quintero.
But popular Malaysian speedster Azizulhasni Awang suffered a disappointing end to his fifth and final Olympics when he was disqualified.
Known as the "Pocket Rocketman" due to his small stature, the 36-year-old won bronze in Rio then silver at Tokyo.
