Hajime Moriyasu's side are looking for a record-extending fifth Asian title and looked set for a comfortable start to their campaign in Qatar when Takumi Minamino gave them an 11th-minute lead.
But Vietnam, who are coached by Japan's 2000 Asian Cup-winning boss Philippe Troussier, turned the game on its head with two goals midway through the first half.
Japan came roaring back before half-time, Minamino scoring a second before Keito Nakamura curled home a sublime effort to send them into the break with the lead.
The second half was a far more sedate affair until Japan substitute Ayase Ueda made the win safe with a fourth goal in the 86th minute.
Japan and Vietnam met in the quarter-finals of the last Asian Cup in 2019, with Japan winning 1-0 before losing to Qatar in the final.
Moriyasu left Kaoru Mitoma out of his matchday squad as the Brighton winger continues his comeback from an ankle injury.
Real Sociedad forward Takefusa Kubo started on the bench but Minamino showed that Japan still had plenty of attacking talent when he opened the scoring early in the first half.
Yukinari Sugawara drilled a ball into the box that deflected off a Vietnam defender and former Liverpool attacker Minamino pounced on the rebound.
Vietnam hit back four minutes later when Nguyen Dinh Bac glanced a looping backwards header over a stunned Japanese defence from a corner.
And more was to come when Vietnam took the lead in the 33rd minute.
Defender Bui Hoang Viet Anh got his head to a free kick swung into the far post and Japan goalkeeper Zion Suzuki could only parry his effort into the path of the onrushing Pham Tuan Hai, who stuffed the ball home.
There was still time for Japan to turn it around before the break.
First, Liverpool's Wataru Endo picked out Minamino in the box and the Monaco forward rolled a precise first-time shot just inside the far post.
Then Nakamura put Japan back in front with a jaw-dropping strike from the edge of the box in first-half injury time.
The tempo dropped significantly in the second half but Ueda notched Japan's fourth with a well-taken strike with just minutes remaining.
Indonesia play Iraq on Monday in the other opening game in Group D.
