On the most recent episode of The Thrive Global Podcast with iHeartRadio, Brandon Stanton, creator of the viral photo-blog Humans of New York, told Thrive Global CEO and founder Arianna Huffington that the hardest part of his work is facing rejection.

Stanton takes portraits of people on the streets of New York and asks them questions about their lives. The result is beautiful photos with heartfelt, deeply personal captions that are relatable yet intimate. But the process behind that finished product, the one we “like” along with HONY’s 18 million other Facebook fans, isn’t as easy as it seems.

“It’s not the people who say yes that make it difficult,” Stanton told Huffington about photographing and interviewing people. Nor is it taking great photos or asking the right questions. There are “thousands and thousands of people who have told me no or rejected me, and then a lot of interviews that didn’t quite work out.”

Stanton said that a huge part of his work is talking to people who “don’t wanna talk to you,” and that people often ignore him. “When people do agree to get into a conversation, a lot of people don’t open up, which is completely their right, but makes it very hard to make a compelling narrative,” he told Huffington. “So there is a lot of work and a lot of rejection behind these seemingly effortless moments.”

What we see as “seemingly effortless moments,” actually include a whole lot of rejection, “thousands of miles walked, thousands of people approached,” Stanton said.

To hear the full conversation, click here.

You can also listen to the Thrive Global Podcast internationally for free on iTunes and Stitcher.