"It was an unbelievable feeling," Guzman said.
"It was a wonderful moment for all of us," Tampa Bay manager Joe Maddon said.
Scott Dohmann (2-0) threw two scoreless innings for the victory.
Sabathia gave up two runs and five hits over eight innings. The left-hander had six strikeouts, giving him 1,101 in his career -- the eighth Cleveland pitcher to reach at least 1,100 strikeouts.
"C.C. was fantastic. Just real good again," Indians manager Eric Wedge said. "Eight strong innings, he gave us every opportunity to win the ballgame."
Sabathia has three consecutive no decisions despite allowing two runs in all three games.
"I just go out and keep trying to keep us in games," Sabathia said. "I'm not really concerned about myself or my record. I just want this team to win ballgames. I just want to pitch in the playoffs."
The AL Central-leading Indians had won three in a row following a four-game losing streak. Cleveland has a 1½-game lead over second-place Detroit, which lost 9-3 to the New York Yankees.
"Our guys are focusing on our game," Wedge said. "We control what we can control. If we go out and execute and step up in RBI opportunities, we're going to score a few more runs and we're not going to make it as difficult for ourselves. That's what we've got to do."
Victor Martinez hit a tiebreaking sacrifice fly in the 10th to give the Indians a 3-2 lead. The Devil Rays pulled even in the bottom half on a two-out RBI single by Carl Crawford off Joe Borowski, who blew his fifth save in 39 opportunities.
"We came over here and took the series with a chance to sweep," Martinez said. "It's always good -- we still took two of three."
Tampa Bay's James Shields allowed two runs and six hits in six innings. He struck out six and walked one.
"Joel Guzman comes up here and had a clutch hit," Shields said. "It was great. Outstanding."
Shields was hit on the back of the right calf by Grady Sizemore's liner in the fourth, but remained in the game after taking a couple warmup tosses.
Martinez put the Indians ahead 1-0 on an RBI single in the first. He has 65 RBI in 68 games when batting fourth in the lineup.
Jhonny Peralta made it 2-0 with a fourth-inning sacrifice fly.
Sabathia retired his first 11 batters before Upton's homer on a 3-2 pitch with two outs in the fourth, cutting the Tampa Bay deficit to 2-1. The shot ended Tampa Bay's homer drought at 291 at-bats.
Upton also made a nice running catch on Sizemore's drive to the center-field warning track in the eighth. He also took an extra-base hit away from Ryan Garko with another running grab for the first out in the 11th.
Pena tied it at 2-2 on his 27th homer of the season in the seventh, equaling a career high set with Detroit in 2004. It was his first homer in 14 games.
- The Indians are off Monday before starting a three-game series Tuesday at Detroit.
- Sabathia moved past Mike Garcia (1,095) into eighth place on the Indians career strikeout list.
- Guzman joined the Devil Rays after Tampa Bay placed SS Ben Zobrist (strained right side muscle) on the 15-day disabled list.
- Pena has hit 20 homers against Cleveland since 2002. Only Paul Konerko of the Chicago White Sox (21) has more.