Artemi Panarin had a goal and two assists, Kevin Rooney scored and Igor Shesterkin made 25 saves as New York snapped a three-game losing streak against Pittsburgh.
''You win a game like this, you kind of get a feeling you are able to do it,'' Rangers defenseman Adam Fox said. ''The guys pulled together and just played a little smarter. It was nice to get over the hump, especially against this team.''
Jason Zucker scored and Casey DeSmith made 21 saves for the Penguins in their fifth road loss this season.
Kreider notched the go-ahead goal at 11:10 of the third period. Fox faked to his left before tossing the puck on net. Kreider redirected the pass past DeSmith. Panarin recorded his second assist on the play and has four multi-point games this season.
''I'm very happy that we finally won after the past three games,'' Artemi Panarin said through a translator. ''Our team did a great job, especially in the third period.''
Zucker opened the scoring midway through the first period. The 29-year-old winger stuffed home a loose puck from a sharp angle after beating K'Andre Miller to the back post. It was Zucker's second goal in as many games. Chad Ruhwedel and Evgeni Malkin assisted on the play.
The Penguins were 0 for 6 on the power play.
''We are not happy with our special teams,'' Zucker said. ''We have to make sure we all get better as a cohesive unit. The simple answer is to shoot pucks.''
The Rangers evened the score 1-1 late in the middle frame on a delayed penalty. Rooney willed a puck into the back of the net after corralling a bouncing puck from Panarin. Defenseman Ryan Lindgren also assisted on the play.
''It looks like we are looking for a better play that's not there,'' Penguins coach Mike Sullivan said. ''We have to do a better job at just putting pucks at the net and then creating our offense that way.''
Panarin added an empty-net goal in the final seconds to seal the victory.
''This was a very important win for our team's confidence,'' Shesterkin said. ''We blocked many shots, put in a lot of work and came out with a positive result.''
Rangers defenseman Brendan Smith left the game in the first period after a collision with Penguins forward Brandon Tanev. Smith did not return due to an upper-body injury.
Anthony Bitetto made his season debut after the Rangers recalled him from the taxi squad earlier in the day.
Defenseman Tony DeAngelo cleared waivers and has played his last game with the Rangers, according to general manager Jeff Gorton. There was reportedly a postgame altercation between DeAngelo and goaltender Alexandar Georgiev after Saturday's game against the Penguins. ''We had an incident that happened,'' Gorton said in a pregame press conference. ''It's in our room. We're dealing with it. This is one of the ways we're dealing with it. Our team is ready to move on.''
NEW YORK GROOVE
The Penguins have two regulation losses in their last 12 trips to Madison Square Garden (8-2-2) dating to December 8, 2014.
The Penguins were originally slated to play the Devils on Tuesday and Thursday, but both of those games have been postponed due to NHL COVID-19 protocol. In addition, the Rangers game against the Devils on Saturday will also be rescheduled.
Rangers: Host Capitals on Thursday.
Penguins: Visit Islanders on Saturday.
More AP NHL coverage: https://apnews.com/hub/NHL
Copyright 2021 STATS LLC and Associated Press. Any commercial use or distribution without the express written consent of STATS LLC and Associated Press is strictly prohibited.