"We hadn't won here for I don't know how long," he said. "That's why it was so good to win here tonight."
Horton scored two goals for the first time since last March to help Panthers beat the Thrashers 5-3 on Thursday and end an 11-game winless streak in Atlanta.
The Panthers, who had gone 0-10-1 at Philips Arena since November 2003, led 2-0 in the first on goals by Horton and Radek Dvorak.
Kovalchuk made it 2-1 in the first minute of the third, taking Todd White's rebound shot and beating goaltender Tomas Vokoun from the left circle.
Atlanta trailed 4-2 when Kovalchuk had a short-handed goal on a breakaway with 5:27 left in the game.
"He's a great player," Vokoun said. "His goals are not by accident. He's smart and he has speed."
Ruslan Salei put Florida ahead 3-1 with 16:51 remaining with a slap shot from the top of the right circle.
Mark Recchi scored his fifth goal in nine games with Atlanta on the power play to make it 4-3.
"Any time you get down in this league, it's so hard to come back," Recchi said. "We battled hard and kept trying to do the right things. Tonight was one of those nights you deserve better."
Horton's second goal, his 10th of the season, put the Panthers up 4-2 after his wrister from the left circle appeared to glance off goalie Kari Lehtonen's glove.
Atlanta lost its second straight to fall two points behind the Panthers and into third place in the Southeast Division. Florida has won four of five.
Martin won his 497th game, putting him in 10th place and one ahead of Glen Sather on the NHL career list. In his third season with Florida, Martin has 90 victories. He also coached 8½ years in Ottawa and two in St. Louis.
But watching Kovalchuk's short-handed goal made Martin furious.
"We had the power play," Martin said. "There's no reason for our defense to pinch. They got the momentum, but we had to stop the bleeding. Even when they scored the first goal, it's 2-1. Then it's 3-1 and then we give them the power play (goal) and it's 3-2. We have get smarter when we have the puck."
Horton made it 1-0 at the 8:16 mark of the first after moving between the crease and left circle and putting his stick on a puck that glanced off referee Gord Dwyer's leg.
Dvorak was credited with scoring his fourth goal and making it 2-0, but Lehtonen seemed at fault after stopping the puck with his glove and accidentally sweeping it past the goal line.
"Some nights you get bounces and some nights you don't," Atlanta coach Don Waddell said. "I thought we were playing pretty well, and all of a sudden we're down 2-0."
Vokoun, who stopped 38 of 41 shots, improved to 6-1-2 in his last nine games, 18-12-2 overall.
Lehtonen dropped to 7-7-0 after making 24 saves.
"You usually get one goal like that every few games, but tonight they got four," Lehtonen said. "Very frustrating. We played well enough to win. Everyone showed up today and played very hard. They were just lucky. They got the breaks."
At least the Panthers could finally leave Atlanta with a victory.
"We hadn't won here," Olli Jokinen said. "We've been missing the playoffs by four or six points, and Atlanta's been taking those points away."
- In his last 21 games, Kovalchuk has scored 17 of his 28 points off goals.
- He has at least two points in 14 of his last 30 games.
- Horton scored two goals in a game for the first time since the Panthers won 5-2 last March 27 at Tampa Bay.