Economic and security issues were at the forefront of talks between the two leaders, as Japan looks to shore up a weak economy and Russia turns to the east to avoid US and European sanctions
Speaking after the initial meeting at the Otani Sanso hot spring resort, Abe said both leaders also discussed their countries' 71-year dispute over the Kuril Islands, to Japan's north.
Abe said he presented letters to his counterpart from former islanders, in both Japanese and Russian.
"We also talked about the possibility of joint economic activities on the islands under a special system and issues related to the peace treaty," Abe said.
The Japanese Prime Minister also expressed his strong concerns over humanitarian issues in Syria and asked Russia to play a constructive role, a high-ranking government official told journalists after the meeting.
Putin reiterated his support for Bashar al-Assad's administration, the official said.
The Russian leader touched down in Abe's home province Thursday afternoon local time, more than an hour late after flight delays.
Speaking ahead of his first meeting with Abe, Putin said the summit talks would help improve bilateral ties between Russia and Japan, according to CNN-affiliate NHK.
He also thanked the Japanese leader for hosting at the hot spring resort, saying there would be "a chance to relax."
He will be staying in Japan for less than two days, including one night in Abe's hometown of Yamaguchi before traveling to Tokyo on Friday, where talks are expected to continue.
Will Putin and Abe finish WWII?
Much of Putin's visit is centered around the little known chain of islands off Japan's northern coast, called the Kuril Islands.
Four of the islands, Iturup, Kunashir, Shikotan and Habomai, were occupied by Russia three days after the end of World War II
, giving them complete control of the Kuril chain.
Japan still considers those four islands part of its territory, a sticking point in relations between the two countries for the past 71 years.
On Thursday evening, Putin and Abe discussed free passage of islanders as well as potential "joint economic projects" to be developed on the islands.
But while Tokyo may be hoping to make progress on having the islands returned, experts told CNN talks were unlikely to yield those kind of results.
"I have a hard time imagining Putin signing an agreement with Japan ceding even a few of the islands because Russian policy is so nationalistic these days," Gerry Hudson, an associate at Ohio State University's Mershon Center for International Security Studies, told CNN in an email.
Trump, Putin make time for Japan
Former US diplomat and adjunct senior fellow at the RAND think tank in Washington, William Courtney, told CNN that Putin's visit to Japan was sending a message to the United States.
"Russia hopes to weaken support for US military basing and deployments, and to undermine public support in Japan for increasing military preparedness," he said.
But Putin isn't the only world leader wooing Japan -- his rare visit comes less than a month after Abe became the first world leader to meet with US President-elect Donald Trump.
Trump and Abe had a "very candid discussion"
in New York on November 17, according to the Japanese prime minister, during what he described as an "unofficial" meeting.
"I am very honored to see the President-elect ahead of other world leaders," Abe told reporters after the discussion
. "The Japan-US alliance is the axis of Japan's diplomacy and security. The alliance becomes alive only when there is trust between us. I would like to build such a trust with Mr. Trump."