“Do you really need to build more good habits? From where I’m sitting you already do a lot of good things. But your bad habits are killing you.”

Shortly after meeting my wife, she hit me over the head with the words above after I mentioned doing some sort of 30-day habit challenge.

At first, I made a feeble attempt to defend my actions. But once she said, “If that hurt a little, I’m probably right,” I stopped talking.

It’s easy to point fingers at other people or make excuses for why we aren’t where we want to be in our lives. Things we can’t control do indeed happen. A great deal of what stops us from making the type of impact we want to make in the world, however, we do to ourselves.

I’ve done some stupid things in my life. Some of these things I did for years before I chose to let them go.

Perhaps the best decision you can make when things aren’t adding up in your life is to take a good look at where you can start subtracting.

This article could be a book. Below are just 4 of the bad habits I kept around for far too long that made my life harder than it had to be.

1. Spending time with charlatans

Legendary investor Charlie Munger, the partner of Warren Buffett, said one of their primary jobs is to identify the people that look good, but are not competent, and remove them from their system.

He may have been speaking in terms of their business, but when it comes to the people we choose to allow into our lives the same logic applies.

I wish I had learned that lesson sooner. I grew up extremely shy with a severe stutter. I used to idolize confident, smooth-talking people.

The only problem was, I failed to take the time to separate those who walked their talk from those who only ran their mouth. I once even trusted a guy to go into business together simply because of his sales skills. That was a stupid decision. Our friendship ended when he literally stole my house.

If you’re impressed by someone, be patient, and do some digging. Watch how they respond when things go wrong. Keep an eye on how they treat other people when they’re stressed out. Take your time when it comes to the people you decide to let into your life. If people continue to prioritize themselves over the people around them, this is a massive sign that they aren’t worth your time.

The biggest mistake not only me, but a lot of people make in life, is they hold low-quality people in high-esteem.

2. Spending money on substances or worthless stuff

Between all the money I’ve spent on alcohol, cigarettes, drugs, and other impulse buys, I could have bought back the house that was stolen from me. And not a small house, but something where my kids wouldn’t have to share a room.

Have fun. Go nuts from time to time. Just don’t be the type of person who complains about life while smoking a Kool in front of Starbucks while sporting a fresh new pair of Weezie’s or whatever they’re called.

This doesn’t mean you have to be a minimalist. I like stuff too. But be smart. Nothing kills more dreams and causes more stress than debt and/or drugs.

Few things are worse than someone who has the potential to make a difference in the world who can’t move their head because they’re either high or because they’re up to their ears in meaningless stuff.

Making the decision to live as clear-headed and lightweight as possible is potentially the best advice there is.

3. Not creating enough white space in my day

Over the last few years, I’ve met some truly interesting people who are doing big things in the world. One of the things that stands out the most to me, however, is they are rarely busy. They always have time for a phone call. They are usually the ones to check in on me to see how I’m doing. I’d be naive to think I am the only person they treat this way.

The people who move the fastest often times live slow boring lives. They take the time to understand what they don’t want to do. As a result, this frees up a lot of time for the things they want to do.

I used to think I was important by always running from place to place. This was a massive waste of not only my time, but more importantly, other people’s time.

If you aren’t taking the time to ask yourself what makes you happy, and creating white space each day to enjoy these things, you can’t blame someone else when things aren’t happening for you.

There will be times in your life when outside circumstances make this difficult. If I’ve learned anything as a father who cares for two young kids while trying to grow my careers it’s that there is always time for the things that mean enough to you. It’s simply up to you to find ways to make it a priority.

The world belongs to the curious — those who have the time to tinker and get lost from time to time.

4. Thinking the world owed me something

I can remember this conversation like it was yesterday. For a year after getting burned by my ex-business partner, at the age of 30, I moped around my parents’ house waiting for the world to be kinder to me again.

My mom, who is quite possibly the most positive and upbeat person in the world sat me down at the kitchen table and said, “The world doesn’t owe you anything. No one is coming to save you. Bad things happen to good people. Don’t waste your life.”

In my early twenties, I was fortunate to start my career in a hot industry. At the age of 23, I was making more money than I deserved. I cursed the man who took that away from me for a long time. I allowed his actions to paralyze me.

Today, however, I see it as the greatest gift I’ve ever received.

Shortly after my mom straightened me out, I took what was left of my money and I boarded a plane to Spain. I wasn’t legally allowed to work in Europe so I had to fight. I worked security jobs, I taught kids, and I ran pub-crawls.

Over time, I got my smile back. That positively attracted the right people. Since then I’ve been doing the work I like surrounded by people I love.

Be a scrapper. Attitude is everything. Everything worth having comes with a few scars.

I wasn’t exactly thrilled the day my wife smacked me in the face with the reality that I was my own worst enemy. But I’m glad it happened.

Nothing holds us back from getting things right than choosing to continue to do the things we know are wrong.

Originally published on Ladders.

Follow us here and subscribe here for all the latest news on how you can keep Thriving.

Stay up to date or catch-up on all our podcasts with Arianna Huffington here.