The Islanders led 3-2 after Noah Dobson's goal early in the third, but Nashville came back to keep New York winless in seven games at its new arena.
Nashville's Yakov Trenin tied it with 7:29 left, and Tolvanen tipped a shot by Roman Josi from the point past Semyon Varlamov.
''It's nice to not only score goals by shooting,'' said the 22-year-old Tolvanen, a first-round pick by Nashville in 2017. ''It's a nice skill to have that I can tip the goals. I practice that a lot.''
The Islanders are 0-5-2 at UBS Arena, which opened with much fanfare late last month. It is the first time in NHL history a team has gone winless in its first seven games in a new venue.
''I felt like we deserved a point, maybe two, but that's not the way it's going right now,'' Islanders coach Barry Trotz said. ''That one hurt because I thought we played a pretty solid game against a good team.''
Varlamov stopped 25 shots but remained winless, falling to 0-5-1.
''I give the Islanders credit. I thought both teams played really well,'' Predators coach John Hynes said. ''It was a hard-fought game, tough environment to play in. But our guys stuck with it and got rewarded for it.''
David Rittich made 18 saves for Nashville, which has won three straight and four of five. The Predators also won the first meeting between the teams this season, 3-2.
Dobson put the Islanders ahead 3-2 at 5:08 of the third when his shot from the right point through traffic eluded Rittich, sending the home crowd into a frenzy.
Trenin tied it by poking the puck past Varlamov for his fourth goal this season.
Rittich stopped Cal Clutterbuck with a close-in shot with two minutes left in the third, before Tolvanen sent the visitors away victorious.
''We've been chasing a lot of games,? Trotz said. ''When you're chasing it, you're taking more risks. I thought our third was fine. I thought we were going to overtime there.''
Islanders forward Austin Czarnik opened the scoring at 6:48 of the first with his first of the season. Czarnik beat Rittich with a short-side shot. The goal was Czarnik's first as an Islander.
Nashville's Ryan Johansen tied it at 17:28 of the first, beating Varlamov for his eighth goal of the season. Johansen, who also assisted on the winning goal along with Roman Josi, has 15 points - three goals and 12 assists - in his last 10 games.
Colton Sissons put Nashville ahead at 18:52 of the first with his third goal of the season.
''We knew they would come out hard and super competitive,'' Sissons said. ''I think we came pretty prepared for that. We remained tough and hung in there and scoring timely goals in the third.''
Islanders captain Anders Lee tied it at 2 with his seventh of the season on the power play at 1:56 of the second, the lone goal of the middle period.
New York was coming off its first win since Nov. 6, a 5-3 victory at Ottawa on Tuesday night, after an 0-8-3 skid.
Islanders defenseman Zdeno Chara played in his 1,627th career game, and the 44-year-old had a spirited fight with Trenin at 13:18 of the first period. The six-foot-9 Chara is third among defensemen in NHL games played behind Chris Chelios (1,651) and Scott Stevens (1,635). Trenin, who is seven inches shorter than Chara, returned after getting seven stitches after the fight.
''That was an unbelivable fight by both guys,'' Hynes said. ''I give Trenny all the credit in the world.''
NOTES: The Islanders scratched defenseman Robin Salo and forwards Matt Martin and Otto Koivula and were still without injured forward Brock Nelson and defenseman Ryan Pulock. ... The Predators scratched goaltender Juuse Saros and defensemen Ben Harpur and Philippe Myers. ... Center Casey Cizikas remained the lone Islander out of the lineup because of COVID-19 protocol. ... Thursday's game was the first in a stretch for the Islanders in which they will play nine of 11 games at home.
Islanders: Hosting New Jersey Devils on Saturday.
Predators: Visiting New Jersey Devils on Friday.
More AP NHL: https://apnews.com/hub/NHL and https://twitter.com/AP-Sports
Copyright 2022 STATS LLC and Associated Press. Any commercial use or distribution without the express written consent of STATS LLC and Associated Press is strictly prohibited.