Federer, a six-time winner in Melbourne, came back from down 8-4 in the fifth-set tiebreak to win 4-6, 7-6 (2), 6-4, 4-6, 7-6 (8).
"All of a sudden you turn the whole thing around within, like, two minutes and it was so worthwhile, you know, everything that I have gone through," Federer said.
"I think if I do play tennis, it's because of winning titles, trying to win as many matches as possible, [enjoying] myself out on court," he added, "but also being in epic matches like this."
The entertaining, back-and-forth contest lasted a tad more than four hours, beginning on Friday and concluding at nearly 1 a.m. Saturday, with roars after each point during the first-to-10 tiebreaker in the fifth set.
Millman was two points away from victory, but the 30-year-old Brisbane, Australia, native saw Federer rattle off six straight points to win the match.
"It came down to the wire at the end. A bit of luck, maybe. I had to stay so focused," Federer said of staring down the man who knocked him out of the 2018 US Open. "He kept on coming up with the goods. ... I was getting ready to explain myself in the press conference."
Federer's biggest issue was his forehand, for so long one of the secrets to his success. It deserted him for stretches, and he finished with a whopping 48 of his 82 unforced errors from that shot.
But that shot also helped him deliver the final winner he would need on match point.
"He pushed me to go for more. You know me: I'm not going to hold back and just rally all the time," Federer said. "I will always try to make plays, and for that I will miss some."
Moments earlier, three consecutive amazing shots -- a backhand stop volley, followed by a pair of forehand passing winners -- by Millman pushed him ahead 8-4 in the final tiebreaker.
It wasn't enough.
"That's what the best players, I guess, do," Millman said of Federer's comeback. "I'll have to go back and watch it."
Novak Djokovic had a much easier time at Rod Laver Arena.
Djokovic, a seven-time Australian Open champion, needed only 85 minutes to beat Nishioka. He had 17 aces and won 43 of 46 points (93.5%) on his first serve, his highest percentage of first-serve points won in a completed major match. At one point, he won 34 consecutive points on his serve.
The win marked the 50th time Djokovic has reached the round of 16 at a major, second among men in the Open era after Federer. He will next face No. 14 Diego Schwartzman, who advanced with a 6-2, 6-3, 7-6 (7) win over Dusan Lajovic.
Cilic, who won the 2014 US Open and was an Australian Open finalist in 2018, defeated ninth-seeded Bautista Agut 6-7 (3), 6-4, 6-0, 5-7, 6-3 to advance to the fourth round.
Bautista Agut beat Cilic in Melbourne last year in the fourth round and also won in 2016 in the third round.
The 39th-ranked Cilic is unseeded at a Grand Slam tournament for the first time since the 2014 Australian Open. He will next play Milos Raonic, who authored a 7-5, 6-4, 7-6 (2) victory over sixth-seeded Stefanos Tsitsipas.
Raonic, a 2016 Wimbledon finalist, had five match points in the third-set tiebreaker and converted his second with a booming forehand winner for the victory at Margaret Court Arena.
Seeded 32nd, Raonic finished with 19 aces and 55 winners in a dominating performance against Tsitsipas, who beat Federer in Melbourne last year on the way to the semifinals.
Raonic's best run in Australia was to the last four in 2016, when he became the first Canadian man to reach the semifinals at Melbourne Park. He lost that match and the Wimbledon final later that year to Andy Murray.
Raonic missed the French and US Open tournaments last year with injuries, but he said he is feeling better after an offseason in which he could focus on recovery.
"It's fun to be healthy, to be playing well," Raonic told the crowd. "I can really take a lot of pleasure in that."
Sandgren, whose best Grand Slam showing was a quarterfinal appearance at the Australian Open in 2018, topped Sam Querrey 6-4, 6-4, 6-4 in an all-U.S. matchup.
Fognini beat No. 22 Guido Pella 7-6 (0), 6-2, 6-3.
"He's a character, man," Sandgren said about Fognini. "What you see is what you get."
The Associated Press contributed to this report.