The former world number one and three-time Grand Slam title winner slipped into retirement when he and Dan Evans were defeated in the men's doubles quarter-finals.
American pair Taylor Fritz and Tommy Paul delivered the knockout blow with a 6-2, 6-4 victory on a packed Court Suzanne Lenglen.
The 37-year-old Murray had already announced that the Olympics would be his last event.
"An incredible competitor on the court," said career-long rival Novak Djokovic on Thursday.
"One of the greatest warriors tennis has seen. His fighting spirit is definitely something that I'm sure is going to inspire many generations to come."
One of the 'Big Four' in the sport, Murray joins 20-time Grand Slam winner Roger Federer in retirement after the Swiss great quit in 2022.
Rafael Nadal, the winner of 22 majors but battling more injuries at the age of 38, exited the Paris Olympics on Wednesday and suggested that he had played his last match at Roland Garros where he won 14 of his Slams.
Nadal also effectively ruled himself out of the US Open, sparking more speculation that the great Spaniard is also finished in the sport.
That would leave just 37-year-old Djokovic, the winner of a record 24 Grand Slams, still active amongst the sport's eminent talents who have carved up 69 majors between them.
Murray famously ended Britain's 77-year wait for a men's champion at Wimbledon when he triumphed in 2013, defeating career-long rival Djokovic in the final.
He added a second title in 2016, taking his career majors total to three after breaking his duck at the 2012 US Open.
Murray won gold at the 2012 Olympics on an emotional day at the All England Club when he defeated Federer just weeks after he had lost the Wimbledon final to the Swiss on the same Centre Court.
Four years later, he defeated Juan Martin del Potro to become the first player, male or female, to win two Olympic singles golds.
Murray also led Britain to the Davis Cup title in 2015, the country's first in 79 years.
He has won 46 titles in all and banked around $65 million in prize money.
However, he has been ravaged by injuries in recent years, slumping to 117th in the world.
The Scot has played with a metal hip since 2019 and suffered ankle damage earlier this year before undergoing surgery to remove a spinal cyst, which ruled him out of singles at Wimbledon.
Instead, he played doubles with brother Jamie and was defeated in the first round before an emotional tribute arranged by tournament chiefs.
"It's hard because I would love to keep playing but I can't," admitted Murray at the All England Club.
"Physically it is too tough now, all of the injuries, they have added up and they haven't been insignificant."
Men's tennis has already opened up a new frontier.
Jannik Sinner, the 22-year-old Italian, succeeded Djokovic as Australian Open champion in January and eventually took his world number one ranking.
Carlos Alcaraz, 21, won the French Open and successfully defended his Wimbledon title, sweeping Djokovic off court in a one-sided final in July.
