Sinner had been due to play singles and doubles at Roland Garros.
His withdrawal means that Novak Djokovic will be the top seed in the singles when the draw is made on Thursday.
A number of players will miss the Paris Games, either through illness or injury.
Later on Wednesday, Denmark’s Holger Rune, the world number 16, announced his withdrawal because of wrist pain.
“I am really sorry that I will be unable to play the Olympics. It’s something I’ve been looking forward to playing and being a part of,” Rune said on X.
Marketa Vondrousova, silver medallist in Tokyo three years ago, pulled out on Tuesday with a hand injury, while Polish world number seven Hubert Hurkacz also withdrew.
Other notable absentees include Britain’s Emma Raducanu, world number three Aryna Sabalenka and two-time Wimbledon runner-up Ons Jabeur.
But Djokovic, Wimbledon champion Carlos Alcaraz and US Open winner Coco Gauff are all slated to play.
The Olympics take place just 13 days after Wimbledon ended and finish three weeks before the US Open begins.