Jay Taylor, Who Has Been Unfaithful in Relationships, Doesn’t Think a Cheater Can Change

• Reality star Jax Taylor cheated on previous girlfriends, including his estranged wife Brittany Cartwright when they dated
• The Vanderpump Rules vet said on his In the Mind of a Man podcast that cheating felt like “a natural thing” to him
• Taylor said he’s “not gonna date” following his divorce from Cartwright due to his tendency to cheat

Once a cheater, always a cheater, according to Jax Taylor.

Taylor — who cheated on his estranged wife Brittany Cartwright before they tied the knot, as well as previous girlfriends — questioned on the Thursday, March 27, episode of his In the Mind of a Man podcast whether “a leopard can change its spots.”

“I’m probably gonna get a lot of flack on this one, but I personally don’t think a cheater can change,” Taylor, 44, said. “Let’s see. So every girlfriend I’ve had, unfortunately, maybe one or two when I was younger, I’ve cheated on. The only time I’ve never cheated is in my marriage.”

The Vanderpump Rules alum recalled cheating on Cartwright, 36, with his SUR coworker Faith Stowers while they were dating. The moment previously played out on the Bravo hit.

“I did have relations with somebody else while I was with Brittany, but we were not married,” Taylor said. “I’m just, I’m gonna be honest, you know. I just don’t think I’m the type of person that can be with one person, you know. The first time I cheated, I just felt like it was a natural thing, which is really f—— disgusting, to be honest with you. So I that’s why I’m not gonna date.”

The reality star referred to himself as “a walking red flag,” “a narcissist” and “a self-sabotager.”

“All the things that I’ve done over the last 14 years, nobody has done to me. I’ve never, never had anybody do anything to me. Everything that I’ve had happened to me, I created,” he admitted. “I’m very, very aware of that. I’m definitely not playing the victim.”

Earlier this month, Taylor opened up about being addicted to cocaine and getting sober from the drug, as well as alcohol. The revelation came just over six months after Cartwright filed for divorce from Taylor following more than four years of marriage.

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

Taylor acknowledged that his substance use drove Cartwright away.

“Instead of just being a man and being sitting Brittany down and being like, ‘Listen, this marriage is just not working out anymore. We should go to therapy or we should take a break.’ That would have been the right thing to do and we wouldn’t have any of these f—— problems that I caused,” he said. “Instead, I decided so let’s make her leave me.”

The Valley star said he doubled up on drinking and substance use so that Cartwright got “so fed up with it that she leaves me.” Taylor called divorce “the hardest thing any man” can go through.

“Losing my father was the worst thing in my entire life. It took me down. It drove me through the mud, but this divorce is just it’s killing me,” the father of 3-year-old son Cruz said. “It’s literally, literally killing me.”

He noted that not being able to see Cruz as much as he’d like was one of the most challenging aspects of the divorce.

“I dug myself in such a hole,” Taylor continued. “And when I mean a hole, like such a big hole that it’s extremely hard to get out of right now. It’s really, really hard. So yeah, I gotta just sum that up as my divorce and I just, I’m really just want this to be over.”

Taylor said he wants Cruz and Cartwright “to be happy,” and he’d advised his son to be the “polar opposite of me” when he grows up.

“I’m relearning myself,” Taylor concluded. “I’m 45 years old. I’ve been on I’ve been on drugs and alcohol for so long that these substances helped me go to events, go to restaurants and be social with people and care what people are actually talking about. And right now, I’m so aware of who I am, and I’m so now the insecurities are starting to come out because I’m seeing them.”