Erik ten Hag's side were in danger of falling victim to an epic giant-killing after blowing a two-goal lead at Rodney Parade.
Bruno Fernandes put United ahead and Kobbie Mainoo's first goal for the club doubled their lead in the first half of the fourth round clash.
But Bryn Morris pulled one back for plucky Newport -- currently 16th in League Two -- before Will Evans rocked United with his second half equaliser.
With United wobbling, Antony was their unlikely saviour as the Brazil forward bagged his first goal since April after a difficult campaign marred by poor form and domestic violence allegations that he denies.
Rasmus Hojlund put a flattering gloss on the scoreline for United when he netted in stoppage-time.
United's victory came on another problematic day for Ten Hag, who had to fend off reports of a row with Marcus Rashford.
Ten Hag, whose side are languishing in eighth place in the Premier League and have been eliminated from the Champions League, explained Rashford's absence from the team by saying he had "reported ill".
However, reports claimed the 26-year-old England forward had been in a nightclub in Northern Ireland on Thursday evening, hours before failing to attend training on Friday.
Asked specifically about that Ten Hag said, "It's an internal matter. I will deal with it."
All the ingredients for a shock were there, given United's spluttering form this term and Newport's past FA Cup heroics.
A raucous crowd, some waving homemade tinfoil FA Cups, crammed into the tiny, ramshackle stadium in the hope of witnessing history.
Temporary seating was added to accomodate demand for tickets, while enterprising fans climbed on scaffolding outside the ground to catch the action.
Newport memorably knocked Leicester out of the FA Cup in 2019, but initially it seemed United would easily avoid becoming another victim of the FA Cup's rich history of upsets.
On the ropes
Ten Hag had taken no chances as he named a strong side featuring all United's key players.
A flowing move produced United's opener in the seventh minute as Luke Shaw's pass was deftly flicked into Fernandes's path by Antony and the Portugal midfielder finished with authority from the edge of the area.
Mainoo doubled United's lead in the 13th minute, sweeping his shot into the far corner from 10 yards after Diogo Dalot picked out the young midfielder.
United looked set to win by a huge margin when Alejandro Garnacho sprinted onto Lisandro Martinez's raking pass and lashed his shot off the crossbar.
But the underdogs reduced the deficit completely against the run of play in the 36th minute.
Morris pounced on a loose ball 25 yards from goal and smashed a superb volley that took a deflection off Martinez before looping past debutant keeper Altay Bayindir into the far corner.
United should have been out of sight by half-time but chances came and went, the most glaring when Nick Townsend plunged to his right to deny Fernandes.
In the 47th minute, Adam Lewis's cross arrowed towards Evans and the forward got there just ahead of Raphael Varane, meeting it with a flick that deflected in off the United defender's boot.
It was a dream come true for Evans, an ex-farmer and teacher who only two years ago was playing in the Welsh leagues.
United were on the ropes but Newport couldn't deliver the knockout blow and Ten Hag's men regained the lead in the 68th minute.
Shaw cleverly dragged the ball back to create space for a shot that cannoned off the post to Antony, who calmly slotted into the empty net.
To Ten Hag's immense relief, Hojlund finished from close-range after a goal-mouth scramble to end any chance of late embarrassment.
