The Supreme Court also ruled that a 20-year statute of limitations on compensation claims could not be applied, a major victory for victims who have endured decades of suffering.
Japan's government acknowledges that around 16,500 people were forcibly sterilised under the eugenics law that was in place between 1948 and 1996.
The law allowed doctors to sterilise people with inheritable intellectual disabilities to "prevent the generation of poor quality descendants".
Another 8,500 people were sterilised with their consent, according to authorities, although lawyers say even those cases were likely "de facto forced" because of the pressure individuals faced.
A 1953 government notice said physical restraint, anaesthesia and even "deception" could be used for the operations.
"I've spent an agonising 66 years because of the government surgery. I want my life back that I was robbed of," victim Saburo Kita, who uses a pseudonym, said before Wednesday's ruling.
Kita was convinced to undergo a vasectomy when he was 14 at a facility housing troubled children. He only confided what had happened to his wife shortly before she died in 2013.
"Only when the government faces up to what it did and takes responsibility will I be able to accept my life, even just a little," Kita, now 81, told a news conference last year.
Apology
The number of operations in Japan slowed to a trickle in the 1980s and 1990s before the law was scrapped in 1996.
That dark history was thrust back under the spotlight in 2018 when a woman in her 60s sued the government over a procedure she had undergone at age 15, opening the floodgates for similar lawsuits.
The government, for its part, "wholeheartedly" apologised after legislation was passed in 2019 stipulating a lump-sum payment of 3.2 million yen (around $20,000 today) per victim.
However, survivors say that was too little to match the severity of their suffering and took their fight to court.
Regional courts have mostly agreed in recent years that the eugenics law constituted a violation of Japan's constitution.
However, judges have been divided on whether claims are valid beyond the 20-year statute of limitations.
Some have said that applying such limitations was extremely cruel and unfair, ordering the state to pay damages. But others dismissed cases, saying the window for pursuing damages had closed.
"The ruling will hopefully pave the way for active steps to be taken by the government to eliminate the kind of eugenic mentality that it created," Kita's lawyer, Naoto Sekiya, told AFP before Wednesday's ruling.
