Ziegler started his career by not allowing a run in 39 consecutive shutout innings -- including the eighth Thursday -- to tie a 59-year-old major league record for relievers. Upton hit an RBI double that scored Akinori Iwamura, who walked, to give the Rays a 5-4 lead in the ninth.
"I didn't think about the streak until I started walking back on the mound," Ziegler said. "The first thought that popped in my head was, 'I can't believe that just happened with a runner on first,' but I guess the good thing it wasn't a cheap one. A whole bunch of things fell at once and so that way there's no pressure from this point on."
Ziegler, Oakland's rookie reliever who began the season in the minors, has been one of the few highlights for Oakland in the second half, setting numerous records including the most shutout innings by any pitcher to start a career. The right-hander extended that mark to tie the major league single-season record for consecutive scoreless innings by a reliever set by Cleveland's Al Benton in 1949.
"It's almost been like a no-hitter type streak or something," A's manager Bob Geren said. "Unbelievable what he's done, but I'll get him right back out and try to start another one. He's been tremendous."
"There were many, many moments where momentum shifted and we were able to stay focused, stay poised and come out of some tough situations," Pena said. "Eventually we were able to put some runs on the board, but the key was being able to stay in the present and not letting that shift of momentum affect our approach at the plate and on the mound. It's a great win for us because it was quite difficult."
Jason Hammel (4-3) pitched 2 2/3 innings to earn the win for Tampa Bay, which lost closer Troy Percival to a sprained right knee in the ninth. Percival hurt himself while fielding a sacrifice by Mark Ellis and had to leave the game after limping back to the mound.
Hammel allowed an unearned run in the 12th before giving way to Trever Miller. Miller the eighth Tampa Bay pitcher, hit Daric Barton before getting pinch-hitter Rob Bowen to hit a grounder to third to earn his first save in two tries this season.
Cliff Floyd walked four times and scored four runs for the Rays, who won two of three from Oakland and haven't lost a series since the All-Star break.
"This type of win is what's been happening since early in the season when we started win a lot of games late," Floyd said. "We've got our confidence up and we believe as long as we go on that field we've got a chance to win."
Tampa Bay missed a chance to win in the 11th inning of the 4-hour, 24-minute game when Rocco Baldelli was tagged out at home plate by Oakland catcher Kurt Suzuki after Casilla's pitch skipped under Suzuki. The ball bounced off the backstop right back to Suzuki, who raced to the plate to get Baldelli.
Pena then hit the first pitch he saw from Casilla in the 12th over the wall in right field to put the Rays ahead for good.
"I can't ask for anything more," Rays manager Joe Maddon said. "It's exactly what we're looking for. Beyond everything else is the continuous maximum effort. It was tremendous. I loved it."
Oakland, which hasn't won a series since taking three of four from Seattle in early July, rallied to tie the game on Frank Thomas' RBI single in the bottom of the ninth but couldn't do much else against Tampa Bay's bullpen.
Carlos Gonzalez hit a two-run homer for the A's, who lost their ninth consecutive series for the first time since 1979. Oakland is 4-21 since the All-Star break, the worst mark in baseball.
Justin Ruggiano tied it at 4 in the fifth with a pinch-hit single to drive in Floyd.
Tampa Bay right-hander James Shields gave up all four Oakland runs and had a season-high five walks in five innings.
A's starter Sean Gallagher had another rough outing, lasting only four innings while giving up three earned runs and walking four.
- Thomas's RBI was the 1,703rd of his career, moving him past Reggie Jackson into 20th place on the all-time list.
- Oakland's Jack Cust tied his own team record by striking out in his 19th consecutive game. Cust, who leads the American League in strikeouts, set the record earlier this year.
- Oakland had not scored more than four runs in its previous 15 games, the longest such streak in Oakland history.