Atkins scored the first seven points for Memphis in the fourth quarter, kickstarting a 17-7 burst that led to an 89-74 lead with five minutes left.
Robert Hite, the undrafted free agent from the University of Miami, tried to bring the Heat back, scoring 16 of his team-high 25 points in the fourth. His breakaway dunk with 28.9 seconds left cut Memphis' lead to 92-88, and his 3-pointer at the horn almost tied it for the Heat (1-3).
Atkins finished with 12 points for the Grizzlies (3-2), as did Stromile Swift and Dahntay Jones. Tarence Kinsey added 10.
Hite led the Heat (1-3) hitting 10 of 14 from the field. Udonis Haslem and Gary Payton each scored 12. Haslem also grabbed eight rebounds and reserve center Earl Barron had 11.
Heat leaders Dwyane Wade and Shaquille O'Neal were given the night off. They made the trip but sat on the Heat bench in street clothes, while Jason Kapono and Alonzo Mourning started in their places.
Memphis is playing through injuries that forced a piecemeal rotation. Though Swift saw his first action of the preseason, starters Mike Miller and Eddie Jones were out, while rookie free agent Kinsey dressed for the first time after recovering from a right orbital fracture.
Both teams shot well in the first half, hitting better than 50 percent until the Heat began misfiring late in the second period, and Memphis built the lead to 42-36 with mostly reserves on the floor.
Memphis held a 44-43 halftime lead. Kinsey missed his first shot, then hit four straight for nine points to lead Memphis, which shot 53 percent.
Although Haslem connected on three baskets early in the second half, Memphis carried a 72-65 lead into the fourth period.