Joseph Young added 25 points for the Ducks (11-0). Calliste, a senior guard, made all 13 of his foul shots.
Tyler Haws led the Cougars (8-5) with 32 points and Kyle Collinsworth added 15 points, five rebounds and eight assists.
Both teams went into the game averaging nearly 90 points. Oregon had a chance to win in regulation, but Elgin Cook missed two foul shots with 1.8 seconds, sending it into overtime tied at 84. Cook finished 1 for 6 on free throws.
Damyean Dotson scored on a drive in the final minute, putting the Ducks ahead for good at 96-94.
BYU led by 10 points in the second half and was still ahead 83-77 before Young's 3-pointer with 1:58 left.
Johnathan Loyd made two free throws with 1:12 left to cut the margin to 84-82. Mike Moser blocked a shot by BYU's Matt Carlino on the next possession, and Moser made a jumper for the Ducks with less than a minute left.
In overtime, with the score tied at 94, Moser drew a charge by Frank Bartley IV with 46.5 seconds left. Dotson scored on the next Ducks possession. Dotson finished with 15 points.
Oregon is 11-0 for the fourth time in school history. The previous time was in 2006-07, when the Ducks made it to the final eight.