this post was submitted on 11 Jun 2023
90 points (96.9% liked)
Asklemmy
43945 readers
627 users here now
A loosely moderated place to ask open-ended questions
If your post meets the following criteria, it's welcome here!
- Open-ended question
- Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
- Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
- Not ad nauseam inducing: please make sure it is a question that would be new to most members
- An actual topic of discussion
Looking for support?
Looking for a community?
- Lemmyverse: community search
- sub.rehab: maps old subreddits to fediverse options, marks official as such
- !lemmy411@lemmy.ca: a community for finding communities
~Icon~ ~by~ ~@Double_A@discuss.tchncs.de~
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
You bring up a very good point. Currently lemmy.ml has thousands of users. Lemmy.world has thousands of users. The hardware they have selected to run their instances is adequate for now, but, what is the plan for scaling out if the user base grows? Is there one? They have a donation page on each lemmy instance (click or tap the heart icon,) but that can’t be enough to pay for the cost of running something used by millions of people, even if only 100s of thousands are ever only online at any given time.
In terms of UI/UX, @dessalines@lemmy.ml has mentioned in a post they are currently working on major performance improvements and enhancements.
It is an old programming trope that premature optimization is a waste of time. As Lemmy scales, several bottlenecks will be hit. Some might be predictable, but many will only become evident after crossing a specific threshold. There are a lot of guides for scaling Mastodon servers after hitting certain bottlenecks, but this is all uncharted territory for Lemmy and we're going to find out the fun way.
Real devs do it in prod!
I’m seriously tempted to write some performance tests in jmeter, locust, or k6, and fire up some live traffic simulations / simulated load against my lemmy instance to see what happens. But at the same time that would feel too much like work and I don’t want to work over the weekend.
Sounds like a great github issue though that we can fund via bountysource or someone with more free time can take a look. Mind creating it?
For an example of a problem that will only come up once it's popular enough, I think hexbear has found that when comment threads get too long (like 800+ nested comments), lemmy starts to break
Ideally, I think no one instance should have a million users to begin with.
User caps might be a good way to decentralize and ensure that we don't end up with just a few mega-instances. If there were a page showing available instances with percent of max users then people could use that when selecting.
Ideally, yes. If that can be the reality, and I suppose that is how it should would with federation, then server costs should never get out of hand.
For that to happen, I believe that interacting with people from other instances and moving your community and account from one instance to another have to become possible / easier.
At present, people flock to the instances with most users as those often have more local content (local content is generally easier to find than federated content) and they often have a smaller risk of shutting down. If I create a community on a smaller instance, the chance of it being found and interacted with are also much smaller than if it had been created on a bigger instance (because of, as I said, local content being user to find).
Sure, I can create an account on myfirstlemmyinstance.com (example URL, not an actual instance) with 10 users, but if my instance decides to shut down, my community of, say, 500 users will now have to move somewhere else and all old content will be deleted.
A “transfer my community” feature that allowed an entire community to be moved between instances would certainly help. That’s a great idea.
From what I’ve seen so far looking through the Postgres db, every instance has data from most other instances. I see users in my local Postgres db from other instances. So, theoretically moving a community from one instance to another could be as simple as changing a few values in the database. Of course in practice it’s never that simple. 😀
Wouldn't it require changing all those values in the database of all instances with subscribers to that community?
Good question. I don’t know. Hypothetically speaking, if the parent instance of the community changes the relevant data in the database to another instance, would federation take over and automatically propagate the change? 🤷🏻♂️
Sounds like an interesting experiment at least, or a possible major bug waiting to happen.
Idk for everyone else, but when I was on reddit once I had set up the subreddits I wanted to see, I really spent 99% on my time on just those. Every so often I would leave or join subreddits but it was rare. Like if people are not doing searches as often then the lag is more tolerable. Plus, won't content from larger and older instances be indexed by search engines eventually? Right now because so many communities are being created on so many different instances, it's more obvious that the searching is laggy but things will surely settle down as time passes.