Then, Gregg Zaun halted New York's playoff charge with one sudden swing.
Zaun homered in the 14th inning and the Toronto Blue Jays regrouped for a 5-4 victory over the Yankees after blowing a four-run lead in the ninth.
"I was proud of how we bounced back in the ninth inning. It's just disappointing how it ended," New York manager Joe Torre said.
Alex Rios hit a two-run homer for the Blue Jays, who won their fourth straight and dropped the Yankees 2½ games behind first-place Boston in the AL East. New York's wild-card lead was trimmed to 4½ games over Detroit.
"Pretty amazing against Doc," Jason Giambi said, calling Halladay by his nickname. "He pretty much was dealing that whole game."
Coming off a three-game sweep of Boston, the Blue Jays played spoiler against New York this time. The Yankees, who had won four in a row, lost for only the third time in 15 games.
In one day, New York also scratched two starters for Saturday.
First, rookie Ian Kennedy was passed over after he felt tightness in his back during a bullpen session this week. Roger Clemens was going to be bumped up one day to pitch in Kennedy's place, but then he was pushed back because his left hamstring is bothering him.
"We're just taking the safe route and giving him a couple days," Torre said.
With one out in the 14th, Zaun drove a 1-1 pitch from Brian Bruney (3-2) to right-center for his 10th homer this season and second RBI of the game.
"I wasn't going to get cheated," Zaun said. "We haven't made too many comebacks in this ballpark, especially after blowing a lead like that."
It was the Yankees' longest game since they played 17 innings on June 1, 2003.
Halladay was in control all night -- until the ninth. Ahead 4-0 and looking for his major league-best eighth complete game, he gave up a leadoff double to Johnny Damon before Derek Jeter grounded out.
Jorge Posada hit a grounder toward the middle that should have ended the game, but Hill bounced a one-hop throw to first that Lyle Overbay juggled. Posada was safe on Hill's error, and Blue Jays manager John Gibbons came out for a brief argument.
"No excuses. I thought I had a grip on it and I didn't," Hill said. "It all worked out I guess, but we could have won it a lot sooner."
Gibbons brought in left-hander Scott Downs, who promptly gave up consecutive hits to left-handed batters. Robinson Cano's single made it 4-3 and Giambi tied it with an RBI single to left-center that sent the crowd of 54,151 into a frenzy.
Halladay initially protested when he was removed, but he also understood the decision.
"Obviously, you're going to be upset when you don't finish it. I felt fine," he said. "But, I don't know how you can second-guess lefty on lefty with the tying run on first base."
Asked how Halladay reacted when he was pulled, Gibbons said: "That's none of your business."
Missing injured sluggers Vernon Wells and Troy Glaus, Toronto snapped a scoreless tie in the seventh with two runs off Wang. Hill hit a leadoff single, went to third on Russ Adams' single and scored when Rodriguez made a diving stop on Zaun's RBI groundout to third.
Adam Lind singled to right and Abreu made an outstanding, one-hop throw to the plate that was in plenty of time. But Posada couldn't corral the ball and was charged with an error after Adams slid in safely.
Rios' two-run shot off Edwar Ramirez in the eighth made it 4-0.
Halladay allowed four runs -- one earned -- and eight hits, striking out four and walking none. He threw 82 of his 110 pitches for strikes.
"That's about as dominant as I've seen him. He was having an easy time with us," Torre said.
Wang yielded two runs, one earned, and six hits in seven innings.
"He was just stacked up against a real good pitcher," Yankees pitching coach Ron Guidry said. "But that's what aces do."
- Halladay has five complete games since the All-Star break.
- Jeter doubled and singled, passing Torre for 118th place on the career hits list with 2,343.
- Yankees reliever Kyle Farnsworth was unavailable because of a tight shoulder.