Jason Spezza also had three points for the Senators, but left in the second with an upper-body injury.
Crosby's 30th goal, a backhand from in close, spoiled Brian Elliott's shutout bid. But Crosby, the NHL's leading scorer with 61 points, also had a couple of costly penalties.
"It wasn't a skill goal, but you just have to get the puck to the net," Elliott said. "I'll take the win any night and I won't care if he scores on me."
Mike Fisher scored the first of Ottawa's two power-play goals midway through the first after Crosby was sent off for hooking. Karlsson made it 2-0 at 13:42 during a delayed slashing call against Crosby, who also hooked Chris Phillips late in the second to negate a Penguins power play.
"It was tough enough as it is, obviously, getting scored on," Crosby said. "Those penalties, I don't know, I wouldn't say I agree with them but that happens in hockey sometimes and probably more times than not guys don't agree with the penalties they get."
Spezza got his third assist on Karlsson's second goal of the game, another power-play effort 20 seconds into the middle period. Spezza left moments later after he was checked into the boards from behind by Kris Letang.
"I saw him going to the puck and he kind of turned his back so I didn't want to hit him, so I put my two hands around him," Letang said. "I think he tripped before and it kind of looked bad on the play."
Elliott turned aside 44 shots, including 22 in the second period, as Ottawa won consecutive games for the first time since a season-high four-game winning streak in early November.
"It was a very good win and we showed that we are still a very good team," Karlsson said.
Marc-Andre Fleury made 21 saves for Pittsburgh, which had won 16 of 18, including three in a row.
"It's pretty clear we had a terrible start and paid for it in the end," Crosby said. "We stuck with things and came at them in the second and third and had some pretty good chances but we dug ourselves a pretty big hole there."
Karlsson got the first of his three points 9:14 in when his shot deflected off Pittsburgh defenseman Paul Martin's stick and struck Fisher on its way past Fleury.
With a delayed penalty called signaled against Crosby, Spezza had time to come off the bench to replace Jarkko Ruutu and get the second of his three assists when Karlsson scored on a shot from the point.
The Swedish defenseman made it a three-goal lead with his eighth of the season early in the second.
"There's always an eye on the fact that you're coming back from two days where you don't skate but that wasn't any different for them and for us," Penguins coach Dan Bylsma said. "It's just a matter of how we're going to play the game and our mentality going in, and in the first we didn't maybe look to execute too much and didn't manage the puck deep a number of occasions and that led to a lot of success for them."
Fleury made a spectacular glove save on Nick Foligno's penalty shot 10:30 into the third. Foligno was hooked on a breakaway before he put a backhand through Fleury but Pittsburgh's Chris Kunitz scooped the puck away barely before it entirely crossed the goal line.
- There was a sold-out crowd of 20,146.
- Phillips played his 900th regular-season game.