Gay had 18 points and 15 rebounds, and DeMar DeRozan added 17 points and a few key baskets down the stretch to spoil Milwaukee's home opener, 97-90 on Saturday night.
''With Rudy rebounding the way he did, his length, his athleticism -- he did an excellent job with that tonight,'' Raptors coach Dwane Casey said.
Gay had another poor game from the field, going 4 of 14 to follow an ugly 6-of-23 performance Friday night in a loss to Atlanta.
He found other ways to contribute, like hitting 8 of 10 from the free throw line and having his big game on the glass.
''I'm just trying not to be one-dimensional,'' Gay said. ''I still wanted to make an effort on defense and the glass ... just make myself be part of the game.''
Toronto snapped a 10-game losing streak to the Bucks, which was the longest winning streak Milwaukee held over any opponent. Toronto's last win was 91-87 on Nov. 1, 2008.
The teams reunited just over a week after an exhibition game between the teams was called off in the first quarter because of slick court conditions.
The Raptors nearly let this win slip away after leading by 12 points to start the fourth quarter.
O.J. Mayo had 16 points off the bench for the Bucks, including a 3-pointer that tied the game at 85 with 5:53 left.
They just couldn't convert on late opportunities, however, including a stretch of 1-of-6 shooting from the free throw line in a four-minute stretch after tying the game. They also missed two 3s in that span.
''I really felt if we just hung around hopefully we could probably try to steal it, but we just didn't make the plays down the stretch,'' coach Larry Drew said.
But the Raptors did, including a step-back jumper by DeRozan to make it 89-85 with four minutes left.
Then after missing two free throws, DeRozan was fouled again after corralling the rebound, making both this time with 2:09 left for a five-point lead.
The Raptors controlled the paint early and outrebounded the Bucks 60-38, and 18-6 on the offensive end. They had a 21-4 edge in second-chance points.
And time and time again, Toronto made it a point to drive the lane and get good ball movement.
''Everybody made a concerted effort to try to get the ball from side-to-side and get everybody involved early,'' said guard Kyle Lowry, who finished with 14 points.
After the ball whipped around the perimeter, DeRozan hit an open 3 from the corner to help build a 77-65 to start the fourth.
But that deficit is nothing for the Bucks, who fell behind by 20 points in each of their first two games on the road, including a rousing 105-98 win earlier in the week in Boston.
True to early season form, Milwaukee stormed back behind frontcourt reserves John Henson (13 points) and Khris Middleton (six), and a 3 from Mayo that tied it at 85 with 5:53 to go.
The buckets stopped falling from there and the defense trailed off.
''Tonight, I thought defensively we weren't tied into together that last four minutes,'' said Mayo, who said he didn't start after arriving late for practice, and apologized for his tardiness.
Caron Butler, a Wisconsin native playing his first home game since being traded to Milwaukee in the offseason, added 12 points and six rebounds.
The game featured all the typical theatrics of a home opener, with a pulsing bass rattling the arena stands and the Bucks walking individually out of the tunnel with the lights dimmed. The team introduced its new coach and 11 new players.
But all that fun was spoiled by the Raptors.
Nate Wolters, a second-round draft pick, made his first career start at the point because of a hamstring injury to Brandon Knight. He finished with seven points and 10 assists.
Besides Knight, the Bucks were also without PG Luke Ridnour (back spasms). ... Raptors G Steve Novak missed his homecoming with a sore back. The seventh-year sharpshooter from the Milwaukee suburb of Brown Deer went to Marquette. He was a freshman on the 2003 team that went to the Final Four.