Just one year after signing him to a lengthy contract extension, the Colts have decided to part ways with general manager Ryan Grigson.

CBS Sports NFL Insider Jason La Canfora has reported that Colts owner Jim Irsay informed Grigson on Saturday that he was being relieved of his duties.

The 44-year-old Grigson had been the general manger in Indy since being hired before the 2012 season. However, Irsay has been growing impatient with Grigson over the last two years, which led to his ousting on Saturday.

Grigson almost got fired after the 2015 season, but was able to hold on to his job after having a sit-down meeting with Irsay. That meeting led to a three-year contract extension that was supposed to keep Grigson in Indy until 2019.

Although the timing of Grigson's firing might come as a surprise, it's not a complete shock that Irsay made the move. The Colts owner didn't give Grigson or coach Chuck Pagano a vote of confidence after the 2016 season, which ended with the Colts missing the playoffs for the second straight year (Pagano did get a vote of confidence at Irsay's press conference to announce Grigson's firing).

Irsay's patience appeared to be wearing thin with Grigson as the season went on. Back in October, the Colts owner defended Grigson after the team's 2-4 start, however, he seemed to lose confidence in his general manager as the season went on. By the time December rolled around, Irsay said he was disappointed at how the Colts' season was playing out.

That December interview was the last time that Irsay talked about the front office. After the season ended, Irsay went dark and didn't give Grigson or Pagano a vote of confidence.

Remember, Grigson is the guy who blamed the Colts' bad defense on Andrew Luck's giant contract. Irsay probably wasn't thrilled when he heard his general manager bashing a contract that Irsay gave out.

"When you pay Andrew (Luck) what we did, it's going to take some time to build on the other side of the ball," Grigson said in October.

So who might be next to fill the GM role in Indy?

La Canfora reported that Irsay actually started the process of replacing Grigson before he fired him. According to La Canfora, Irsay had been "covertly" looking around for a new general manager over the past few weeks.

Irsay's search wasn't as covert as he thought, though. During the search, word leaked that the Colts owner had approached Peyton Manning about taking over the general manager role or another front office position.

If Irsay were able to convince Manning to return to Indy, it would put Manning in a position of building a team around the quarterback who replaced him.

At this point, though, Irsay said that Manning likely won't be the hire this time around.

"I wouldn't say that he's in the picture," Irsay said of Manning on Saturday. "Him and I have talked. He's not in the mix for this process, interviewing general managers."

If the Colts decided to promote someone from within to replace Grigson, La Canfora reports that personnel executive Jimmy Raye could be their man. The Colts confirmed on Saturday that Raye will be interviewed for the job.

As for team reaction, Grigson's firing seems to be a popular move so far. Colts punter Pat McAfee had this to say after hearing the news.

McAfee isn't the only one who seems to dislike Grigson. The former general manager never seemed to be able to get along with Pagano.

Pagano and Grigson have been at odds for several years, which made it somewhat of a surprise last season when Irsay gave them both extensions that would've kept them together through the 2019 season. The good news for Pagano is that he doesn't have to deal with Grigson anymore and now, the Colts can hire a GM who will actually get along with their head coach.