It was their fifth Grand Slam title as a team.
Mahut fell on his back and put his head in his hands after they became the first all-French team to win twice at the clay-court major since World War II.
Mahut and Herbert won their first title in Paris in 2018.
A very emotional Mahut held back tears and said winning the gold medal in doubles at the Olympics could well be the final big goal of their career.
"We are working very hard to achieve this," he said.
Bublik and Golubev were bidding to win their first tour-level title and to become the first Kazakh men to win a Grand Slam title in any discipline. The only Kazakh player with a major title is Yaroslava Shvedova, who won the women's doubles titles at Wimbledon and the US Open in 2010.
The Kazakh pair served for the match at 5-4 in the second set, but Bublik was broken after hitting a double fault. Mahut and Herbert then dictated play in the tiebreaker. They saved two break points in the fourth game of the decider and broke for a 4-3 lead.
The French fans rendered an a cappella version of France's national anthem to celebrate their win.