It’s easy to assume that creativity is something that comes from within. Something we posses in large or small measure. That’s how science saw it for decades. But more recent research shows how our creative abilities grow or shrink based on the situations we find ourselves in and on the people around us. Surprisingly, the friends and co-workers who make us feel most validated and at home may be draining our abilities to forge new creative paths and make bold changes.

Why should this be? Don’t people who make us feel safe allow us to take risks and express our full creative gifts? This is true to some degree. Researchers at Berkeley found that when groups of diverse people discussed the value of political correctness before trying to solve a problem, this setting up of a safe space increased their creative abilities. Making everyone feel valued as human beings within the group allowed them to disagree more vocally and take more risks with their ideas.

In this study, the time they took to build safety before they got unsafe worked well with diverse groups of strangers. But with close friends and collaborators, we often drift toward getting safer and safer until, without realizing it, our creativity plummets.

Consider this: Of the “Big Five” traits psychologists commonly use to describe someone’s personality, none is quite so appealing as “agreeable.” Here are just a few of the lovely characteristics of highly agreeable people: they are trusting, easy to get along with, cooperative, sympathetic, and selfless.

Of course, most of us want agreeable friends and collaborators. High schoolers tend to rate their agreeable peers as more likable, while bosses prize their most agreeable employees as team players. But here’s the problem: When agreeable people get together, they often wind up focusing more on making each other feel good than accomplishing anything of value. Researchers at the University of Richmond found that teams made of highly agreeable people not only performed worse on creative business tasks than more cantankerous teams but often couldn’t tell when things were going wrong. They judged their satisfaction with their team based on how good it felt to be together rather than what they were getting done. Teams with a few disagreeable individuals showed a different pattern. Not only did they perform better, but their satisfaction tended to track well with their results. Feeling good came not from getting along but from making progress, and so they had a functional internal compass to guide them to success.

If you find your friends and colleagues to be highly agreeable, there’s a good chance that you’re facing subtle pressure to maintain good feelings over expressing your full creative gifts. It might be time to add a few disagreeable types to your circle.

Here’s a second important recent finding: groups of people who share the kinds of life experiences that lead to shared political and social values don’t get as creative as groups that deeply disagree on important matters.

A study from the University of Michigan and Loyola University showed that groups selected at random tend to be more creative at solving complex problems in fields like business, public policy, and education than even handpicked groups of the best and brightest experts. That’s because experts tend to have similar backgrounds and life experiences.

“Where everyone thinks in the same way, everyone will get stuck in the same place,” explains one of the researchers Scott Page. “But if we have people with diverse tools, they’ll get stuck in different places. One person can do their best, and then someone else can come in and improve on it.”

These findings should throw up warning signals for the growing number of workplaces that proudly declare that they hire for cultural fit over skills (Google calls their cultural fit Googliness). Without a doubt, we want to keep chronically negative, destructive personalities off our teams, but how quickly can cultural fit come to mean people who think like we do and employ the same mindsets? And the same goes for friendships. We want people who care about what we care about. We often choose our friends because they agree with us on the most important political and social issues that help us define our identities. But can we really understand our world if we only connect with people who pretty much agree with us on everything?

Here’s what all this science adds up to: When we think about getting out of our comfort zones in our thinking, the unexpected first step may be getting out of our comfort zones in our relationships. Spend more time with people that disagree with you, that see the world differently and that like to challenge you as much as support you and you’re likely to see your creative output skyrocket. Sure, creativity comes from within. But that’s only half the story. It’s the factors outside of you may that are likely to be the unexplored territory that will unlock more of your creative genius.

Excerpted from Unsafe Thinking: How to be Nimble and Bold When You Need It Most by Jonah Sachs. Copyright ©2018. Available from Da Capo Lifelong Books, an imprint of Perseus Books, LLC, a subsidiary of Hachette Book Group, Inc.