The world No. 1 was beaten 6-4 6-3 by a player to whom she had never lost in their four previous meetings.
Williams, who had already been knocked out of the doubles tournament along with sister Venus, equaled Steffi Graf's open era record of 22 grand slams last month by winning Wimbledon.
But while the 34-year-old was favored by many to add a second Olympic singles title, she was unable to find a way past a determined opponent.
"It's obviously disappointing but, you know, she played really well and I think the better player won today," Williams said.
"I can't wait for the next time and I know next time it's going to be a really good match. I'll look forward to it.
"It didn't work out the way I wanted to, but at least I was able to make it to Rio. That was one of my goals."
Svitolina, ranked 20th in the world, was beaten 6-1 6-1 by Williams in their last meeting, on clay, at the French Open in June.
But she showed little sign of being overawed on this occasion, unleashing a number of impressive ground strokes on the Rio hard courts to leave Williams in trouble.
The Odessa-born player looked like romping to victory after breaking her American opponent early in the second set.
But Williams broke back to level at 3-3 despite being hampered by a number of unforced errors.
Double faults continued to haunt her in the next game as she served five of them to gift Svitolina a 4-3 lead.
Williams missed last month's Rogers Cup in Canada with inflammation of the shoulder, and here she looked uncomfortable throughout the second set.
Trailing 0-40 at 3-5, Williams saved the first two match points as the pressure began to build.
But she could not find a way to escape on this occasion, going long on a forehand to confirm her fate.
Svitolina will next play two-time Wimbledon champion Petra Kvitova. The Czech reached the quarterfinals by beating Ekaterina Makarova of Russia.
Williams was not the only big name to be ousted Tuesday as French Open champion Garbine Muguruza
suffered a crushing 6-1 6-1 defeat by Puerto Rico's Monica Puig.
Puig, who had never beaten a player in the top five before, eased past the world No.4 to set up a quarterfinal clash with Germany's Laura Siegemund.
"I consider it the biggest win of my career because it's at the Olympics," the 22-year-old told reporters.
"This happens once every four years. Right now, I'm in the clouds. I can't even believe it because truly it was incredible."
There were no such problems for second ranked Angelique Kerber after the German defeated Australia's Sam Stosur in straight sets.
She will play Britain's Johanna Konta who overcame Russia's Svetlana Kuznetsova 3-6 7-5 7-5.
American Madison Keys is still in contention for a medal after seeing off Spain's Carla Suarez Navarro in three sets.
She will take on Russia's Daria Kasatkina for a place in the last four of the competition.