Story highlights

The announcement comes days before the games are set to start

Security concerns have mounted recently

Rio de Janeiro CNN  —