Though the Hawks couldn't clinch the final Eastern Conference berth because the Knicks won earlier in Toronto, Atlanta remained two games up with three to play. The Hawks need just one more win or New York loss, and can secure their postseason place by beating Miami at home on Saturday.
Atlanta scored the final six points after Brooklyn took an 88-87 lead with 2:22 left.
"It says a lot about our team, says a lot about what we're trying to accomplish here," Millsap said. "The focus that it took to lock in, compete and to get this win, it says a lot. Guys want to win, guys want to try to make this push."
Teague added 22 points for the Hawks, who won for just the fifth time in 13 games and are only 36-43, but that might be good enough in the East.
He dominated his matchup against younger brother Marquis during a first-half stretch that turned the game and couldn't control a grin when he crossed Marquis over so badly in the fourth quarter that it caused him fall, but didn't want to talk about playing against his brother afterward.
"I'm always aggressive. That's how I've got to play," he said. "Kyle (Korver) pulled me to the side and said I've got to be aggressive for us to be a good team and that's how I've been playing lately."
Atlanta was just 1 of 19 from 3-point range, but hit 20 of its 21 free throws.
"Man, it was great. We need to start another one," Nets guard Joe Johnson said. "I'm glad this streak is over with. Let's start another one for another month and a half, two months."
Paul Pierce had 13 points for the Nets, becoming the 18th player in NBA history with 25,000. But he was just 2 of 6 from the free throw line and 1 of 7 behind the arc.
"It's better to be in the championship club," Pierce said. "Obviously, statistical things come and go. There's going to be players in the future that's going to pass me up. But you know when you win, that lasts forever."
He became the fourth active player and second on the Nets in the 25,000 club, joining Kobe Bryant, Dirk Nowitzki and longtime teammate Kevin Garnett.
"I told him welcome to the neighborhood, man," Garnett said.
The Hawks opened their largest lead at 64-53 midway through the third quarter after a run of seven straight points, but the Nets made 10 of 16 shots in the period and were back within 74-73 heading to the fourth.
Plumlee's three-point play gave Brooklyn an 88-87 lead with 2:22 to play. Millsap's reverse layup gave Atlanta the lead again, and the Hawks got a break when an original offensive foul call on Teague for charging into Jorge Gutierrez was overturned when instant replay showed Gutierrez was in the restricted area. The free throws made it 91-88 with 1:46 remaining, and the Nets couldn't score again before Teague put it away with two more free throws.
Brooklyn led 23-16 after one but the Hawks turned around the game with a 39-point second quarter, when Jeff Teague scored 14 points in 7 minutes.
Much of that came when he was against Marquis, a seldom-used player in his second season and first with the Nets after he was traded this season from Chicago. Marquis Teague was forced to play extended minutes when Gutierrez picked up his third foul with 5:40 left in the half and the Nets leading by two.
Jeff Teague, about 4 years older, then scored 11 points in the final 5:16, converting a three-point play when his brother fouled him and later making two free throws after beating Marquis so badly with a crossover dribble that Marquis had to foul him to stop a layup.
There was a moment of silence before the game for Lou Hudson, the former Hawks All-Star who died Friday at 69. ... Williams was hosting 65 families of children affected by autism in luxury suites. Williams, whose son is autistic, was scheduled for a meet and greet session with the families after the game.