I've been doing some thinking lately and I've come to the conclusion that one of the biggest hurdles for TTRPG newbies is comfort level. Most people just aren't that comfortable opening up to other people, even friends or family, on a creative and imaginative level. I've been doing it for twenty years and I still get a little nervous when I try to speak like the NPCs or my character. I have a very hard time concentrating on playing a role-playing game when there are non-participants in the room, or even in the adjacent room if the door is open.
Role-playing is actually a very intimate thing when you consider it - you're opening up your heart and soul and sharing your dreams with people with the expectation that they're going to listen and react and build upon your thoughts and feelings. This takes a huge amount of trust and confidence. Most people just lack that confidence and trust in their friends (as sad as that sounds, I believe it's true).
There's some inborn fear that people will look down on you or laugh (at least on the inside) when you share an idea or make some kind of story suggestion or speak in character or whatever. It's borderline playing pretend - big kids, teenagers, and adults have all been conditioned to leave childish things like imagination behind. And this is on top of the inherent difficulty of improvisation and the awful social stigma that D&D has earned over the years. What if I say something dumb? What if everyone else is much more creative then me? Or worst of all... what if someone "cool" hears me talking about dragons and magic and knights and make-believe crap?!
In general, people are much more comfortable enjoying, referencing, discussing, and playing with "creative stuff" that come from known, established, and respected creative sources. Even the worst movie script ever will get a lot of attention if it's coming from a famous director or writer. I could come up with the greatest song in the world, but it might never get any attention because people don't know, care about, or have any respect for my musical talent - why would they - I'm a nobody! In fact, many of the songs I have written and played for people, they assumed were covers because people are naturally more skeptical and judgmental of creativity that comes from people they know and consider themselves equal with (implying that most people put famous artists on a pedestal). This trails into the whole peacock feathers thing - people often like famous people because there must be something to like if everyone else likes them.
I think that deep down, people really do like the idea of role-playing games, but want a less intimate way to interact with their friends where they don't have to put their imagination on the chopping block. Card games, board games, and video games serve this purpose deftly. With these games, you still get to tap into the part of your brain that enjoys imagining and experience another world, but all of the creative stuff is out of your hands. It's way less intimate to interact with other people on a rules-level because the rules are explicit and they come from known, trusted, respected, pedestalled sources. Rules, numbers, dice, and cards are sterile and therefore safe for interaction between you and your friends. Sure, I might use a poor strategy, but the mistake was fundamentally impersonal because it existed within the realm of rules created by a game designer
So what does this have to do with anything? When it comes to the future of the hobby, a lot. Designers have wracked their brains for decades in an attempt to find a way to bring D&D to the mainstream but have failed every time because ultimately the general populace doesn't want role-playing games. They have other forms of entertainment that are way more socially acceptable, easy to figure out, and probably most important, impersonal. So what do you do about that?
Well, you can attempt to worm TTRPGs into the growing mainstream geek community by trying to make them resemble other types of more-socially-acceptable games like card building games or board games, but that has been done before with little significant success. Actually, it seems the only way to turn a game on to the mainstream to strip away the things that make a TTRPG until there is nothing left that could be recognized as a role-playing game. In other words, once you take out the self-generated story and other creative elements of the game that come from the imagination of the people playing, it suddenly becomes much easier for people to play..
Instead, I think the focus needs to draw away from publicity and mainstreaming, and turn towards making games for the existing audience. Keeping the existing players interested in the hobby is more important than gaining the occasional flash-in-the-pan visiting gamer. Make games for the people who are already buying them and make it easy for them to introduce this hobby to their friends. That's what we need to be focusing on.