The landslides buried dozens of homes, and floodwaters inundated thousands of houses, said Sutopo Purwo Nugroho, spokesman for the Indonesian Disaster Management Agency.
Of the 35 killed, 31 died in landslides and four died in the flooding. An additional 25 people remain missing, the disaster management agency said.
Heavy rains drenched several areas for much of Saturday, causing flooding and landslides in 16 towns and cities across the province.
The Bogowonto River in Purworejo also swelled due to floodwater.
Most of the landslides and flooding occurred in the Purworejo subdistrict, where 11 people died and several are reported missing.
In one deadly incident, truck passengers tried to clear debris from a small landslide that had blocked the road. Numerous motorcycles were behind the truck when a bigger landslide hit them. Nine people died.
Disaster officials, the military and police joined forces with nongovernmental agencies and volunteers to rescue and evacuate residents from affected areas.
Unusually heavy rains
Heavy rains and flooding also hit West Sumatra on Thursday and Friday, killing one person and displacing thousands. The floods there have since subsided.
June is usually relatively dry in the region, but the National Climatology, Meteorology and Geophysics Agency has issued a warning for potential rain for late June and early July in five provinces.
High tides and waves are also predicted for the southern coast of Sumatra, Java and East Nusa Tenggara.
Nugroho said the weather anomaly is the result of several factors -- including warm sea temperatures and the massive flow of moist air from the Indian Ocean -- converging in Sumatra and Kalimantan, resulting to unstable atmospheric conditions and increased rainfall.