The hosts, who came in undefeated into the title clash with 10 wins, were bowled out for 240, a total Australia overhauled in 43 overs with Travis Head smashing 137 in Ahmedabad.
The result shattered India's dreams of ending a global title drought stretching back to the 2013 Champions Trophy.
"I mean the result hasn't gone our way and we know that we were not good enough on the day. But proud of the team," Rohit said after the loss.
"It wasn't supposed to be. Honestly, 20-30 more runs would've been good. I thought when KL (Rahul) and Virat (Kohli) were batting, we were looking at 270-280 at that point and we kept losing wickets."
Rohit hit a quickfire 47 before Australia hit back with wickets and bowled out the hosts in 50 overs despite fifties from Virat Kohli (54) and KL Rahul (66).
Australia slipped to 47-3, but the left-handed Travis Head stood firm and put on 192 runs with Marnus Labuschagne, who made an undefeated 58.
"With 240 on the board, we wanted early wickets but credit to Travis Head and Marnus," said Rohit.
"They put us completely out of the game and I thought the wicket got better to bat under lights. I mean we knew under lights, it would be better but don't want to give that as an excuse."
