/kbin meta

2 readers

1 users here now

Magazine dedicated to discussions about the kbin itself. Provide feedback, ask questions, suggest improvements, and engage in conversations related to the platform organization, policies, features, and community dynamics. ---- * Roadmap 2023 * m/kbinDevlog * m/kbinDesign

founded 1 year ago

lemmy.ml is no longer shadowbanning kbin. (lemmy.ml)

submitted 1 year ago* (last edited 1 year ago) by blightbow@kbin.social to c/kbinMeta@kbin.social

79 comments fedilink hide all child comments

Apparently one of the lemmy.ml admins was overzealous in banning all User-Agent strings that contained the word "bot". Bans were entered for all of the individual strings containing that word which were observed in their webserver logs, which impacted kbin's reported agent of "kbinBot".

The issue has been fixed, and I observed that one of my kbin posts to a lemmy.ml community was successfully pushed to the original instance.

Edit:

Here are all the links that I've found with the lemmy.ml admins discussing the issue:

https://lemmy.world/comment/1082149 (appears to have been where they first noticed and fixed the issue)
https://lemmy.ml/post/1920972 (posted a few hours later, provides additional context for what happened)

you are viewing a single comment's thread
view the rest of the comments

[–] blightbow@kbin.social 1 points 1 year ago* (last edited 1 year ago)

There are plenty of high volume, non-malicious bots that do. robots.txt is a thing for a reason, and we can see here that lemmy.ml has implemented it. Not all bots that ignore robots.txt are malicious though, just poorly designed. You can basically lump them into three categories:

Well-behaved bots that announce that they're bots in the User-Agent header and obey robots.txt (note that they may still slam the server even if they obey it)
Mediocre bots that announce that they're bots in the User-Agent header but ignore robots.txt (or vice versa)
Bad bots (malicious or otherwise) that announce their User-Agent as other things, often pretending to be other software.

Their logs told them they had a lot of traffic from stuff identifying itself as bots. Throwing that traffic out wouldn't break lemmy but would help them deal with the capacity problems that all of the mainstream lemmy/kbin instances had to deal with shortly after the Reddit exodus began. They fucked up and tagged kbin in the process, which definitely would have been one of the highest volume ActivityPub consumers matching their criteria.