The Reds outfielder connected on a two-run homer in the 10th inning, and Cincinnati held on to beat Pittsburgh 9-8 in a battle between the worst two teams in the National League.
Dunn's homer off closer Matt Capps (4-5) came an inning after Jeff Keppinger's solo homer sent the game to extra innings.
"(Capps) is really good, but it happens. That's baseball," said Dunn, whose team hit three home runs. "We're swinging the bats pretty good right now. It's a funny game. You've got to take the good with the bad, and hopefully the good lasts a whole lot longer than the bad does."
After Brandon Phillips led off the tenth with a single, Dunn connected on his 30th homer of the season into the right-centerfield seats.
"Once the ball hit the bat, it was gone," Capps said. "It's a tough feeling.
"Today I let the guys down and let the team down, blew the save and ultimately got the loss."
David Weathers gave up pinch hitter Jack Wilson's solo homer in the bottom of the 10th before finishing for his 21st save.
Jason Bay, Nate McLouth, Ronny Paulino and starting pitcher Matt Morris also homered for the Pirates, the first time they've had five homers in a game since August 2003. Pittsburgh, which has won only four times since the All-Star break, squandered leads of 6-4 and 7-6.
Edwin Encarnacion also went deep for the Reds. Jared Burton (3-1) earned the victory with 1 2/3 innings of scoreless work.
Morris, who was acquired in a trade from the San Francisco Giants on Tuesday and instantly became the highest-paid player in club history, was in line to earn the win when he left the game in the seventh.
The veteran allowed five runs -- four earned -- on nine hits and a walk, striking out four. He also had two hits, including his second career home run.
"I really put us in a hole, but with some timely hits we battled back and had a two-run lead, and we just couldn't hold it there," said Morris, who has never lost at PNC Park. "But if keep playing like that, well get wins."
He led off the bottom of the third by taking a 1-2 pitch from Bobby Livingston into the first row of seats in left-center, near the deepest part of the ballpark, an estimated 400 feet.
The Reds tied the game at 1 in the fourth on Ken Griffey Jr.'s RBI single.
Cincinnati took the lead with a three-run sixth. Encarnacion's two-out single scored Norris Hopper. Javier Valentin, who went 3-for-5, followed with a double just out of the reach of diving left-fielder Bay, driving in Phillips and Encarnacion.
Livingston retired 10 of the 12 batters he faced after the Morris home run, but was pulled with one out and a runner on in the sixth. His replacement, Todd Coffey, immediately gave up the two-run homer to Bay, his 17th, cutting the Reds' lead to 4-3.
After Coffey hit Jose Bautista and walked Paulino, Mike Gosling came on and surrendered McLouth's homer to give Pittsburgh a 6-4 lead. It was McLouth's fourth of the season.
Livingston was charged with two runs on six hits and no walks while striking out four in 5 1/3 innings.
"Livingston gave us a great outing," Reds manager Pete Mackanin said. "He did a great job, everything we would want him to do."
Hopper made it 6-5 the next half-inning when he doubled, stole third and came home on catcher Paulino's high throw.
Paulino's solo shot in the eighth put Pittsburgh on top.
"When they got that run, it didn't take the wind out of our sails," Mackanin said. "We just kept going."
Keppinger, a former Pirates farmhand, was an unlikely source of power. The 180-pound middle infielder came into the game as part of a double switch an inning earlier.
"Our team scores close to 50 percent of our runs on home runs," Keppinger said. "And we've got some big guys that swing the bat, so that's what they do and that's what they did tonight -- and lucky for us, we came out with a win."
The last Pirates pitcher to homer was Kip Wells on August 19, 2005, in Philadelphia. ... It was Morris' 10th career multi-hit game, the last a three-hit contest June 1 in Philadelphia. ... Morris' other homer was April 16, 2003, off of Wayne Franklin in Milwaukee. ... As an opponent, Morris was 6-0 with a 2.34 ERA in eight career starts at PNC Park. ... Pirates catchers have not thrown out an opposing base-stealer in their past 19 attempts. ... The Pirates' Adam LaRoche stole his first career base in the ninth.