Osasuna' five-man defence held out until the 87th minute when Yannick Carrasco delivered a perfect cross from the left and Felipe jumped between two defenders to head into the net.
The win put fourth-placed Atleti on 26 points, two points behind leaders Sevilla, who were held 2-2 at home by Alaves.
Atletico struggled to establish control as Osasuna slowly grew into the game and gave the home side a really hard time.
Jagoba Arrasate's team were comfortable in possession and equally relaxed dealing with Atleti's attacks.
Atleti coach Diego Simeone rested several players including Luis Suarez and Rodrigo de Paul but brought both on in the second half in a desperate bid to breach Osasuna's defence.
"My reflection is what you saw, a team that protected itself well, they defended extraordinarily which is why they've won so many games away from home," Simeone said. "We knew that, and we looked to get in behind, creating difficulties to get the ball into the box, finishing quickly.
"There weren't many clear chances because the opponent defended very well. The goal was going to come from a mistake, a shot, a rebound, they're a team that works extraordinarily well in defence."
The hosts created very few opportunities until Felipe struck to finally give the home fans a chance to celebrate.
It was a goal of redemption for Felipe, who was in the eye of the storm after Atletico's Champions League defeat at Liverpool earlier this month when he made mistakes that led to both goals before being sent off for a foul on Sadio Mane.
"It's an unforgettable night. It was a long time since I scored and it is very special for me," Felipe told reporters.
"We worked really hard not to suffer because suffering is part of Atletico's history. We talked all week trying to correct some mistakes but it was a very complicated game."
Simeone added: "It was important for the team and for [Felipe], and for his confidence. He's played quite a lot since his arrival, sharing the position. I'm happy for him. With his physique in attack he's hard to control, and the delivery of the corner was great, it was a real golazo."