Gadhafi was released from the northwestern Libyan city of Zintan under a "General Amnesty Law" passed by Libya's House of Representatives, the Abu Bakr al-Sideeq militia said in a statement Saturday.
He has not been shown in public.
A close associate of Saif told CNN Gadhafi was released Friday but would not reveal Gadhafi's current location, citing security concerns.
Gadhafi is wanted by the International Criminal Court for alleged crimes against humanity.
According to Human Rights Watch, he hasn't been seen or heard from since June 2014, when he appeared via video link at a trial session
being held in a Tripoli court over the suppression of the 2011 Libyan revolution.
"The reported release of Saif al-Islam Gaddafi based on the Libyan parliament's 2015 flawed amnesty law does not change the fact that he is wanted by the International Criminal Court on charges of crimes against humanity related to the 2011 uprising," said Sarah Leah Whitson, Middle East and North Africa director at HRW. "Libyan authorities, who remain obligated to surrender him to the court, should urgently confirm whether he was released and disclose his current whereabouts."
Libya has been divided since Moammar Gadhafi's fall
nearly six years ago, with three governments, multiple parliaments and competing tribes and militia vying for power and influence. ISIS also operates in the North African country.
The House of Representatives -- based in the country's east -- is not internationally recognized.
However, in its statement, the Abu Bakr al-Sideeq Brigade said Libya's House of Representatives was the country's legitimate authority and it was releasing Gadhafi as a result of that parliament's amnesty.
"Based on that we have released Mr Saif al-Islam Moammar al-Gadhafi and he is free. We confirm that he left the city of Zintan on the 14th of Ramadan ... we call on all rehabilitation and correctional facilities to follow Zintan and release all the political prisoners who are included in the amnesty law."
'Protected by the Libyan people'
Saif al-Islam Gadhafi's lawyer, Khaled ElZaidy confirmed Gadhafi's release, telling CNN: "He is free in a safe place in an undisclosed location in a Libyan city."
ElZaidy said that he expected Gadhafi to address the Libyan people.
"Unlike the different governments, the will of the people is where he gets his power from," the lawyer said. "He is protected by the Libyan people, the tribes and the people who are his incubator ... he is popular and there is no worry about him. In every part of Libya he is protected by the Libyan tribes."
ElZaidy said Gadhafi had been following developments in Libya during his captivity and that he wanted to work on reconciliation and fighting terrorism in Libya.
"His priority is to eradicate terrorism, to bring security then bring back life and economic prosperity," he said. "Any international organizations that want to combat terrorism, will find Saif Gadhafi. He will have a major role in bringing peace to Libya."
Former heir apparent
Gadhafi was once considered the heir apparent to his father, who ruled Libya for over 30 years.
Before the 2011 Libyan civil war, he was believed to be a moderate in comparison to his father.
But in 2011 the International Criminal Court issued a warrant for his arrest for alleged crimes against humanity during attempts to quash the revolution that led to his father's ouster.
Libyan authorities refused to hand him over, saying the court in The Hague did not have authority to try the case.
A trial was held in Libya's capital, with Gadhafi facing charges relating to attempts to suppress the revolution uprising, including the killing of protesters -- a crime punishable by death.
The UN High Commissioner for Human Rights criticized the trial, asserting that it failed to meet international standards for fair trials.