One of those put to death was Yasutaka Fujishiro, 65, who murdered seven people including his aunt and neighbors in 2004, according to Japan's Justice Ministry.
The other two, Tomoaki Takanezawa, 54, and Mitsunori Onogawa, 44, killed two employees in pachinko (game) parlors in 2003, the Justice Ministry confirmed.
"These are extremely brutal cases, taking precious lives for selfish reasons. I think these are terrible incidents not only for victims who lost their lives but also for bereaved families," Justice Minister Yoshihisa Furukawa said at a news conference.
All executions in Japan are carried out by hanging. Prisoners often learn of their execution just hours before it's due to take place. Their families are usually notified about the execution only after it is over, according to rights group Amnesty International.
Japan's use of the death penalty -- and the manner in which it is carried out -- has long angered rights groups and campaigners working to abolish the practice.
"The recent appointment of Prime Minister Fumio Kishida was a chance for progress on human rights in Japan. But today's abhorrent resumption of executions is a damning indictment of this government's lack of respect for the right to life," said Chiara Sangiorgio, death penalty adviser at Amnesty International, in response to Tuesday's executions.
"After two years without executions, this feels like a missed opportunity for Japan to take long overdue steps to abolish the cruel practice of the death penalty."
Japan is one of only a handful of nations that continues to use capital punishment -- among industrialized democracies, only parts of the United States use the practice. Amnesty has repeatedly called on Japan to establish an immediate official moratorium on all executions as a first step toward total abolition.
In November, two death row inmates launched a lawsuit against the government, demanding it change the practice and seeking compensation for the impact of the "inhumane" practice, according to Reuters.
Japan has resisted calls to change and there are many in the country that support the death penalty.
"The abolition of the death penalty is an important issue relating to the foundation of the Japanese criminal justice system," said Deputy Chief Cabinet Secretary Seiji Kihara at a news conference. "It is not easy to decide on the death penalty, but considering that these crimes are still going on, I don't believe it is appropriate to abolish the death penalty."