The defender became the most expensive player in Bundesliga history when he joined Bayern from Atletico Madrid in 2019 for 80 million euros ($90 million).
But the 25-year-old said a string of injuries isolated him from his teammates, and he also had to deal with judicial problems in Spain.
"They weren't easy years," Hernandez said in an interview with German magazine Kicker.
"I was injured for the first year and a half and it was definitely the worst moment of my career.
"First I missed six months, then I played two months, and then I had four months out again.
"When you're not with your teammates every day and you're alone often, it gets you down. But now that I am with them I feel well integrated into the team."
In October, Hernandez appeared before a Madrid court for violating a restraining order in 2017 banning him from contacting his partner after a violent altercation between the pair, who are now married.
Neither made a complaint against the other but both were charged by Spain's public prosecutor.
Hernandez avoided a sentence after the court accepted the French player's appeal.
"It was a very difficult time for me when this story came out. My wife was, and she still is, pregnant with our second child. Neither my wife nor I understood why the court opened this procedure against me.
"It was an incident which went back to 2017, it's the past, the present is that my wife is pregnant, that I love her and she loves me."
However, the defender conceded that the enforced absences strengthened his resolve.
"Some players when they are injured think about stopping football. For me it was exactly the opposite.
"The bigger the difficulties, the more I'm motivated to get through and back to work. These moments made me stronger mentally and more mature from a character point of view."
