Rutherford went missing during a visit to Afghanistan as a tourist.
A video released by the Taliban in May 2011
purported to show a man who identified himself as Rutherford answering questions about himself in English.
CNN was not able to independently confirm the video's authenticity when it was released.
"It's my understanding that the government has done all that they can do and they've just not been successful. So, now the family has to take it upon themselves to try to bring Colin home," Rutherford's mother, Wendy, told CNN affiliate CBC News in 2011.
His brother, Brian, told CBC that the family was willing to negotiate.
"Let's do something to make that happen," Brian Rutherford said. "Please contact us."
It's not clear how exactly Rutherford earned his freedom -- whether a ransom was paid or a prisoner-swap was negotiated. The government of former Canadian Prime Minister Stephen Harper held a policy of not paying ransoms.
The current Canadian government "does not negotiate with, or pay ransoms to, terrorist organizations," government spokeswoman Rachna Mishra told CNN.
"Canada is very pleased that efforts undertaken to secure the release of Colin Rutherford from captivity have been successful," Dion said. "We look forward to Mr. Rutherford being able to return to Canada and reunite with his family and loved ones."
Dion also thanked the government of Qatar for its help in freeing Rutherford. Qatar has a history of mediating negotiations
between Western nations and terrorist organizations.