Matteo Darmian nodded Inter ahead in the fifth minute before Hakan Calhanoglu made sure that the majority of the around 70,000 fans at the San Siro went home happy with his 70th-minute penalty.
Simone Inzaghi's team are level with three other teams on four points, including Parma and Genoa who were 1-0 winners at Monza, after their first two matches of the new campaign.
"Winning is hard but staying up there is even harder. We had to bounce back after drawing at Genoa and we managed to do that," said Darmian to Sky Sport.
"Our secret is who we have in the squad. At Inter there is a lot of competition and that internal competition is good for everyone."
Inter were far from their thrilling best but didn't need to be against sorry Lecce who are pointless, have conceded six and are yet to score after two fiendish opening fixtures against the champions and Atalanta.
Without injured captain and star striker Lautaro Martinez, Mehdi Taremi made his first start for Inter since signing last month after being released by Porto.
The Iran forward had a hand in Darmian's opener, getting a slight touch on a cross and diverting the ball to the Italy defender who made no mistake with an angled header.
Taremi also had claims for a penalty waved away shortly after the break following a clash with Kialonda Gaspar in a bit of penalty box chaos, which led to Marcus Thuram having a close-range shot saved by Wladimiro Falcone.
It was Thuram who earned Calhanoglu the chance to seal the points from the spot when as he challenged for a cross Gaspar bundled over the France forward, and Turkey midfielder Calhanoglu confidently stroked home.
Milan 'problems'
Matteo Cancellieri pushed home the decisive goal 13 minutes before the end of an entertaining encounter at the Stadio Ennio Tardini in which Parma defied their newly-promoted status against leaky Milan.
The 22-year-old winger perfectly met Pontus Almqvist's low cross to score his first goal since being loaned to Parma by Lazio, who lost 2-1 at Udinese.
Milan have collected one point from their first two league matches under Portuguese Fonseca, whose team looked set to go on and take the win when in the 66th minute Christian Pulisic levelled Dennis Man's early close-range finish.
They didn't really wake up until after the half-hour mark when they tested Zion Suzuki three times in the space of a few minutes but and even though Milan were mostly on top they were repeatedly cut open by Parma's counter-attacks.
"I can't hide that it's my responsibility, but it seems clear to me that we have a problem with defending and being aggressive," Fonseca told DAZN.
"We had a good pre-season, but the truth is that this is where we are."
Parma were dangerous every time they went on the counter-attack and could have been out of sight by half-time had their finishing not been so wayward.
Coach Fabio Pecchia celebrated his 51st birthday with a brilliant win which put his team on four points.
"We took on Milan with personality and always had in our heads the idea that we could win," Pecchia told reporters.
"I want to thank the boys because they gave me a great present today."
On Sunday, Antonio Conte's Napoli try to bounce back from a humiliating opening defeat at Verona when they host Bologna, while Roma face Empoli with talisman Paulo Dybala whose expected move to Saudi Arabia fell through.
