Even so, Millsap was proud that the Hawks held together despite the absence of their leading scorer and rebounder.
"Everybody's still a little shocked about the situation," Millsap said. "You could tell. Al's a big part of what we do, but once we shook it off, guys just got out there and played ball."
Playing their first game since Horford suffered a torn pectoral muscle, the Hawks trailed by 14 after Henderson hit a 20-foot fadeaway jumper with 4:19 left in the third quarter.
But Atlanta, which has won two straight and five of six, rallied to force overtime when backup center Pero Antic made a one-legged 3-pointer from the right wing with 3.1 seconds left in regulation.
"It's going to take some time to learn to play without Al," Hawks coach Mike Budenholzer said. "We have to get everybody comfortable, to keep the ball moving, to know where the spots are. I thought our start was OK, actually."
The Bobcats, who have lost two straight and three of four, didn't trail again in regulation after Ben Gordon's 3 from the left corner made it 86-84.
Antic's shot while moving away from the basket made it 101-all and the game went to overtime when Henderson's fadeaway jumper bounced off the rim at the buzzer.
Jefferson bemoaned that Antic was able to get the ball in a scramble and score his only points of the game.
"My guy hit a shot fake, going left, off one leg, a 3-point shot to tie the game," Jefferson said. "I mean, what can you say? It's part of the game, I guess. It hurts, though."
In the final minute of OT, Millsap hit a pair of free throws to make it 108-all, Kemba Walker missed a wild shot for Charlotte and Williams hit two foul shots to give Atlanta the lead for good at 110-108.
Williams made all 10 foul shots in overtime.
There were five lead changes in the first half and five in the second half. Charlotte's biggest advantage before intermission was 10 on Henderson's 4-foot turnaround jumper with 4:22 left in the second quarter.
Walker, who scored at least 20 in six straight games before finishing with 18 in Friday's home loss to Oklahoma City, missed his first six shots before hitting a 3-pointer that put Charlotte up 55-47 early in the third.
Walker finished 4 for 20 from the field and scored 16 points.
"I just told them, we have to get past the point where everybody's happy that the Bobcats aren't getting beat every night," Charlotte coach Steve Clifford said. "Because we're better than that, OK? We can play better than this."
It took until the final seconds of the third for Atlanta to cut the lead to six when Antic blocked three shots by Cody Zeller on the same possession and Williams followed with a 3 from the left sideline to make it 77-71.
"We did a great job of getting stops there at the end," Williams said. "We made timely baskets. We made big shots in situations where we needed them. In overtime, we just battled."
Atlanta G Kyle Korver hit a 3 early in the fourth to extend his NBA career record to a 99th consecutive game. ... With Horford sidelined, Millsap and G DeMarre Carroll are the only Hawks to start and play in all 30 games. Budenholzer said before the game that the team doesn't know much time Horford will miss. He tore his right pectoral muscle in Friday's OT win at Cleveland. Two years ago, Horford played in just 11 regular-season games after tearing his left pectoral muscle. ... Carroll was a game-time decision with a sprained right thumb and was scoreless after going 0 for 4 from the field. ... Charlotte dropped to 6-4 on the road against Eastern Conference opponents. ... The Bobcats dropped to 5-14 at Philips Arena. ... Horford's replacements at center -- Elton Brand, Antic and Gustavo Ayon -- were a combined 2 for 13 from the field for six points.