Savinho and Erling Haaland struck either side of half-time as City ended a run of eight away games without a win.
The performance was still far from the standards that Guardiola's side have set in winning an unprecedented four consecutive English top-flight titles.
But the effusive celebrations of Haaland's header 16 minutes from time showed that three points was all that mattered for the visitors to at least temporarily halt their remarkable slump.
"It is not enjoyable right now, it is just relief. We have done incredible things for many years," said Guardiola.
"Our mood is in a better position. It was tight in the second half, they were much better.
"We don't have energy to sustain 90 minutes. Hopefully the new year can help us bounce back."
Victory lifts City up to fifth but they are still 11 points behind leaders Liverpool, who have two games in hand.
Defeat leaves Leicester still rooted in the bottom three.
Guardiola made just one change from the 1-1 Boxing Day draw against Everton as Kevin De Bruyne replaced his Belgian international colleague Jeremy Doku.
De Bruyne's fitness struggles have played a part in City's slump and he immediately showed what Guardiola's men have been missing for most of the season.
Rico Lewis picked out De Bruyne, who cushioned a cross into the path of Haaland but his low effort was well saved by Leicester's stand-in goalkeeper Jakub Stolarczyk.
Leicester have now lost their last four games after a bright start to Ruud van Nistelrooy's reign.
The Foxes were left to rue not taking their chances to inflict more misery on a City side still showing a clear lack of confidence.
Any time the home side crossed into City territory in the first half they appeared a major threat.
Jamie Vardy would surely have won a penalty had he not strayed offside before being brought down by Stefan Ortega, while James Justin's header hit the post after a goalmouth scramble.
City, though, got the crucial opening goal on 21 minutes.
Stolarczyk should have done better than to parry Phil Foden's long-range effort into the path of Savinho, who swept the rebound into the roof of the net for his first City goal.
Chances continued to come and go for Leicester early in the second half.
Justin should have levelled but his mishit finish from close range allowed Manuel Akanji to clear off the line.
Vardy then had the biggest chance to level when he prodded over Stephy Mavididi's brilliant in-swinging cross.
However, they were hit by a City sucker punch to end the defending champions' miserable run on the road.
Savinho was the creator this time as his cross perfectly picked out Haaland who powered home his 19th of the season, ending a four-game goal drought for the Norwegian.
"It's always a relief when you score, it was an important goal to go 2-0 up," said Haaland.
"It's been a difficult time, now it's New Year's -- five or six days till the next game. It's the longest time in a long period so we can rest a little bit."
