He must repay money spent on renovations unrelated to security, the Constitutional Court said. The National Treasury will determine the amount he must repay.
"The constitution, rule of law and accountability is the sharp and mighty sword ready to chop off the ugly head of impunity," the court said.
The decision caps a controversy that dates back nearly seven years, when Zuma took office for his first term.
Renovations to his home started soon after and included a swimming pool, cattle enclosure, chicken run, visitors center and amphitheater.
Opposition parties filed two cases alleging misuse of public funds over the hefty price tag.
Zuma: 'Will reflect on judgment'
After the ruling, a statement from the South African government said Zuma "has noted and respects" the judgment.
"The President will reflect on the judgment and its implications on the state and government, and will in consultation with other impacted institutions of state determine the appropriate action," the statement added.
The court also found the country's National Assembly in violation for its actions regarding the investigation of the President.
"The Court thus held that the National Assembly's resolution, based on the minister's findings exonerating the President from liability, was inconsistent with the Constitution and unlawful," the ruling summary said.
Two years ago, an investigation
by independent watchdog Public Protector found
that Zuma spent 246 million rand ($15 million) to renovate his home in Nkandla.
The Public Protector asked Zuma to pay "a reasonable percentage of the cost" for upgrades not related to security, which would be determined by the National Treasury.
Zuma never repaid, saying the findings were merely recommendations and not legal court orders. A police inquiry into his house cleared him of wrongdoing, saying the upgrades were made for security reasons.
But the top court said Thursday that not all are security-related, and it ordered the National Treasury to determine the amount he must repay.
The Treasury has 60 days to file a report detailing the amount, and Zuma has 45 days after that to pay the money.
The scandal has added to public discontent with Zuma, who's made recent headlines for the wrong reasons, including the dismissal of Finance Minister Nhlanhla Nene.
Nene's firing came after the National Treasury refused to approve several controversial deals and was interpreted as the President's inclination to remove anyone who stands in his way.
The deals rejected included one between South African Airways and Airbus, and one to fund a 1 trillion rand ($50 billion) nuclear deal with Russia.
Repayment offer rejected
When things started spiraling, Zuma offered to repay some of the amount, but opposition parties pressed forward with their cases.
"Zuma must go down. Zuma must leave the office of the President," said Julius Malema, leader of the Economic Freedom Fighters, one of the parties that took him to court. "The President, the executive, has failed. The parliament has failed."
Malema said his party would disagree with any payment settlement without reinforcing the powers of the Public Protector.
Opposition leader Mmusi Maimane
said the ruling marks "a significant day for South Africa" and called it an opportunity to begin the impeachment process against Zuma.
"It affirms the separation of powers, that the judiciary certainly can make judgments of this nature without fear of prejudice," Maimane said. "But the second thing is that actually the President, as we've always maintained, always acted outside the prescribe of the law and it is a great opportunity now, we believe, to begin the process of impeaching the President."
Zuma, the country's fourth post-apartheid president, was sworn in for a second term in May
He spent most of his life in Nkandla, which is about 150 miles (240 kilometers) north of Durban
Complaints about the renovations go as far back as 2009, according to the Public Protector's report.
At the time, the funds in question totaled $4 million. But the upgrades progressed, and the amount ballooned to $15 million.