Lee scored with 2:44 left after Pittsburgh's Teddy Blueger was penalized for delay of game. It was the fourth goal this season for the Islanders captain. Ryan Pulock and Mathew Barzal had assists.
Semyon Varlamov made 28 saves for the Islanders, who improved to 4-4-2.
''It was a big win for us because we stopped a little bit of a slide here,'' Islanders coach Barry Trotz said.
Jake Guentzel put the Penguins ahead 3-2 at 3:19 of the third before Cal Clutterbuck tied it for the Islanders with his first goal of the season at 8:23.
Penguins captain Sidney Crosby assisted on Guentzel's fourth goal of the season. Crosby has 114 points against Islanders, the most of any active player against one team. Tristian Jarry made 22 saves for the Penguins, who fell to 5-5-2.
Jordan Eberle opened the scoring for the Islanders with a slick backhand goal at 10:22 of the first period.
Pittsburgh tied it when rookie defenseman Pierre-Oliver Joseph scored his first NHL goal, beating Varlamov with a shot from the left circle at 13:43. Crosby immediately scooped up the souvenir puck for his 21-year-old teammate, who was acquired by Pittsburgh in a trade that send forward Phil Kessel to Arizona in June 2019.
''Playing with great players like I am able to is surreal,'' Joseph said. ''It would have been better to come away with the two points.''
Eberle scored his second of the game and fifth of the season at 15:45 of the first. He was happy to see his teammates forge a victory in the closing minutes after this frustrating road trip.
''Going into third period, every guy in the locker room knew we needed to get two points,'' Eberle said. ''In this short season, you don't want to get too far behind. We need to get on a roll and get back into it.''
Michael Dal Colle had assist on both Eberle goals, his first two points of the season.
The Penguins had the better of play in the second and finally tied it for a second time when Evgeni Malkin benefited from a turnover in the Islanders' zone and buried the puck past Varlamov at 19:44.
It was the second goal this season for Malkin, who has 31 goals and 48 assists in 65 career games against the Islanders. Only Crosby has more points against the Islanders among active players.
Both teams were coming off a week of two postponed games each because of COVID-19 precautions. Pittsburgh's two home games - scheduled for Feb. 2 and 4 against the New Jersey Devils - were postponed by the league, and the Islanders' two home contests against Buffalo were similarly postponed.
''I think there were a lot of positives to take from it. I thought we controlled the third period. There was a lot to like of our overall game,'' Penguins coach Mike Sullivan said. ''We have to make sure we don't give our opponents easy opportunities to score.''
The Islanders had last played on Sunday, a 4-3 overtime loss at Philadelphia to finish an 0-3-2 road trip. Their previous victory was a 4-1 home win over the Devils on Jan. 21.
''We were able to get a good bounce,'' Lee said of his winning goal. ''It was a big one for us.?
Pittsburgh's previous game was a 3-1 road loss against the Rangers on Monday. The Penguins are on a seven-game road trip, their longest since an eight-game trip in 1997.
The Islanders are 4-1-0 when scoring first.
They will next face the Rangers on Monday at Madison Square Garden, where he teams split the opening two games of the short season with shutouts in both games.
''The best way to get on a run is your best players being your best players night in and night out,'' Trotz added. ''We're not there yet but we will be. I trust this group. We just have to get a little traction.''
Varlamov played his 500th career game, becoming the 76th goaltender in NHL history to reach that mark.
Trotz has 849 victories, tied with Ken Hitchcock as third-winningest coach in league history behind Scotty Bowman with 1,244 wins and Florida's Joel Quenneville.
The Islanders have three players with consecutive game streaks of at least 240 games with Anders Lee 278, Brock Nelson 255 and Mathew Barzal at 242.
Islanders: Visit the New York Rangers on Monday.
Penguins: Visit the New Jersey Devils on Tuesday.
More AP NHL: https://apnews.com/hub/NHL
Copyright 2021 STATS LLC and Associated Press. Any commercial use or distribution without the express written consent of STATS LLC and Associated Press is strictly prohibited.