Ok, I've been setting up a bunch of remote subs and I think I have a better idea of the behavior we're seeing here. I should note, I don't know what's SUPPOSED to be happening and I don't know what happens on other Lemmy instances. But I can now see what's happening here on lemme.world and it's consistent but complicated enough to be confusing.

Basically, it seems like the community list at https://lemmy.world/communities is some kind of local cache and it seems to get populated whenever someone searches for a sub with the bang syntax, like [!mastodon@lemmy.ml](/c/mastodon@lemmy.ml). So you end up with a confusing sequence of events like this:

• Time 0:
  • James visits https://lemmy.world/communities to look in the ALL list for mastodon@lemmy.ml, but it's not listed.
  • James does a keyword search for mastodon in the community searchbox, the mastodon community is not returned because it's not in this instances community list yet.
  • James gives up, reports that the mastodon community is missing.
• Time 1:
  • Janet visits https://lemmy.world/communities and searches for [!mastodon@lemmy.ml](/c/mastodon@lemmy.ml) using the bang syntax. No results are returned.
  • In the background, Janet's search causes our instance to discover the community and silently add to the community list. I don't know why this happens, but it happens reliably.
  • Janet gives up, reports that the mastodon community is missing (even though she actually just added it to the list, she didn't check again to realize that).
• Time 2:
  • Tom visits https://lemmy.world/communities and sees mastodon@lemmy.ml is already there. Reports everything is fine, other people must have been confused or that replication kicked in on its own (even though Janet JUST added it to the list, he didn't see the timing correlation to realize that Janet's search triggered the community being added to our list).
  • Tina does a keyword search with mastodon, and it returns the community she's looking for.
  • Janet searches using the bang syntax like [!mastodon@lemmy.ml](/c/mastodon@lemmy.ml), STILL gets no results even though the community is in our community list. Edit: Since writing this I've been told that lemmy searches that return zero results may actually just not be finished. Maybe I/Janet would have gotten a result here if we waited long enough. Or maybe would have eventually gotten a result even at Time 1.)

So you can see that you'll observe different behaviors based on whether you do a keyword search, bang-syntax search, or browse the community list... and you'll see different results depending on whether ANYONE has searched using the bang-syntax yet.

So the reliable way to subscribe to a community is first to search by bang-syntax, then scan the community list (or keyword search) AFTER the bang-search... and then you'll find your community.

No idea if this weird behavior is by-design, a bug in Lemmy itself, or a problem with the lemmy.world setup. But I'm pretty sure it explains all the weird behaviors we've seen around communities sometimes being easy to find and other times being hard to find.

I don't have insights into how the communities get discovered, though I'm new to the ecosystem too. If I find anything as I'm reading around I'll pipe up. In the meantime I'll try other ways of searching as you've described. Get some rest, cheers.