Gustavsson, 50, leaves after four years in charge.
"We thank him for his strong contribution, passion and commitment during that time and wish him every success for the future," Football Australia chief executive James Johnson said in a statement.
The process to find Gustavsson's replacement would "commence immediately", Johnson added.
A 2-1 Olympic defeat to the United States sealed the Matildas' earliest exit from an Olympic Games since their debut in 2000.
It is a sharp fall from grace for the Australian team which finished fourth in Tokyo three years ago.
Under Gustavsson's stewardship, Australia had also reached the semi-finals of last year's Women's World Cup, which they co-hosted with New Zealand.
Australia lost their opening game in Paris 3-0 to Germany and despite bouncing back with a 6-5 win over Zambia, defeat to the United States sealed their exit.
Their faint hopes of scraping into the quarter-finals as one of the best third-placed teams ended when Canada beat Colombia 1-0.
Defending champions Canada went through instead despite being docked six points for a spying scandal.
Canadian head coach Bev Priestman was sent home and hit with a one-year ban from football after a member of her coaching staff used a drone to spy on a New Zealand training session before their opening match.
"It has been a great honour and privilege to have been able to be the head coach of the Matildas over the past four years," Gustavsson said in a statement.
"Australian football will be forever in my heart, and I will be watching on and cheering on your success in the future."
