Why Scott Hamilton Is Choosing Not to Treat His Third Brain Tumor: 'I'm at Peace with the Decision' (Exclusive)

Ice skating legend Scott Hamilton has had to face his fears many times in life, from learning to skate as a little kid to dealing with the grief of losing his mother to breast cancer in 1977. He also faced testicular cancer in 1997, from which he successfully recovered after surgery and chemotherapy.

He had to face his fears again in 2016, when he learned that a pituitary tumor in his brain had returned — for the third time. But unlike he had in years past, Hamilton felt confident in his decision not to treat it.

"When they gave me the diagnosis, they said, it's back," he tells PEOPLE. "And so they brought in this guy, a really young, talented surgeon, and he said, 'We could do the surgery again. It'd be complicated, but we've got really talented people here that we could bring in, and I know we could pull it off if that's an option for you.'"

But Hamilton says that the entire time the surgeon was talking, his mind was elsewhere. He'd already gone through surgery twice, first when he was diagnosed with a brain tumor in 2004, and then again when it returned in 2010.

The second time around proved to be extremely complicated, resulting in nine different surgeries after an artery in his brain was nicked. Hamilton knew he didn't want to go through a similar ordeal.

"All I felt was just, don't worry about this. Just go home and get strong," he says. "They go, 'Well, what do you want to do?' And I said, 'I think I'm going to go home and get strong.'"

He didn't know exactly what he meant by that. "I was just answering my spirit," he says.

The decision proved to be a good one. "It's been remarkable," he says. "I went back to the scan three months later and they said, it hasn't grown. I go back three months later and they go, it shrank 45%. I said to my surgeon, 'Can you explain this?' And he said, 'God.' I went back in, and it shrunk 25% again."

However, the next time he returned to get it checked, the news wasn't as positive.

"It had grown," he says. "And then COVID hit and going into any kind of hospital situation was almost impossible. So in my spirit, in my inner being, I realized, I'm totally at peace with not even looking at it again unless I become symptomatic," he says.

He understands not everyone would be able to comprehend this plan – and that's not to say he won't treat it eventually, when it's needed.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from celebrity news to compelling human interest stories.

"The ace I have up my sleeve is that now there is a targeted radiation therapy that will shrink the tumor," he says. "And in that, I can avoid a lot of other things like surgery and chemo. So I don't know, I'm mostly trying to be in the moment and taking all the information and do the right thing when the time comes."

Hamilton, who turned 65 this year and is currently celebrating the 40th anniversary of his gold medal win at the 1984 Sarajevo Olympics, says his life's achievements have truly exceeded all expectations.

'I'm blessed beyond my wildest imagination," he says, noting that his Scott Hamilton CARES foundation has funded cutting-edge cancer research that has saved countless lives, and that the Stars on Ice tour, which he helped launch in 1986, is still wildly popular today.

"I never would've thought to dream any of the things that have happened to me," says the father of four.

"I never would've thought to dream that one day I would found a cancer organization that's actually going to have impact and save lives. I never would've thought to dream that an Olympic gold medal experience would've allowed me to give so much back to my sport and help create a platform to give careers to so many of the greatest skaters in the history of the sport."

"And to have my children and just how amazing they are, and my wife and how amazing she is? I never would've thought to dream any of it."

Correction: An earlier version of this story indicated that Hamilton's brain tumors were cancerous. In fact, all three of his brain tumors have been benign. Hamilton has only been diagnosed with cancer once, in 1997, when he had testicular cancer.