this post was submitted on 04 Nov 2024
33 points (94.6% liked)
Piracy: ꜱᴀɪʟ ᴛʜᴇ ʜɪɢʜ ꜱᴇᴀꜱ
54565 readers
473 users here now
⚓ Dedicated to the discussion of digital piracy, including ethical problems and legal advancements.
Rules • Full Version
1. Posts must be related to the discussion of digital piracy
2. Don't request invites, trade, sell, or self-promote
3. Don't request or link to specific pirated titles, including DMs
4. Don't submit low-quality posts, be entitled, or harass others
Loot, Pillage, & Plunder
📜 c/Piracy Wiki (Community Edition):
💰 Please help cover server costs.
 |
 |
|---|---|
| Ko-fi | Liberapay |
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Why not run your own scraper?
https://bitmagnet.io/ (probably the simplest)
https://github.com/seik/stilio
https://github.com/AlphaReign/scraper
https://github.com/alanshaw/libp2p-dht-scrape-aas
https://github.com/boramalper/magnetico & a cli search utility for it: https://sr.ht/~rakoo/magneticos/
That's really cool!
I have a few questions
How much configuration is required so that it successfully finds all the high quality torrents?
How long do I need to keep it running to find existing quality torrents?
How resource intensive is the program?
I have a hitch there must be extra steps so make the data reliable but who knows ,
Is that one definitely scraping bittorrent DHT? From what I can tell it looks like it used for scraping IPFS DHT via HTTP. Been a bit since I tinkered with IPFS but I do know services exist to scrape the IPFS DHT, this might be one of the backend tools being used for that.
Is this significantly different from the QBitTorrent search engine?