565
Twitter silently removes login requirement for viewing tweets
(techcrunch.com)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 05 Jul 2023
565 points (98.3% liked)
World News
32076 readers
935 users here now
News from around the world!
Rules:
-
Please only post links to actual news sources, no tabloid sites, etc
-
No NSFW content
-
No hate speech, bigotry, propaganda, etc
founded 4 years ago
MODERATORS
They actually likely did this due to SEO. Google was allegedly in the process of removing tweets from the search index because they weren’t accessible. This happens automatically for most sites.
How does Pinterest get around this then? They pollute image searches like crazy, and require you to login to see anything. At least they did, I blocked them from searches so maybe it's different now.
Easy - detect if you're getting accessed by a search crawler or a human. Serve a full page or just a login request.
So how can a user pretend to be a web crawler?
This firefox addon can spoof useragents:https://add0n.com/useragent-switcher.html
You're going to need a special hat.
Adding a clipboard and a ladder will make it even more official
Ever heard of https://12ft.io/ ? It allows you to bypass alot of pay walls by basically pretending to be a search engine trying to index a website. For SEO reasons a lot of pay walled sites allow search engines to access the whole article to index. 12ft.io leverages this to show you whole articles behind paywalls. This is something you could also achieve by spoofing the User-Agent. It would probably work for things like Pinterest without an account as well, but that's something I have never tried (since I have no interest in the cancer that is Pinterest).