1188
this post was submitted on 05 Oct 2023
1188 points (98.2% liked)
Technology
59243 readers
3352 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
This is a copy/pasted message I wrote up on another thread. As long as there are people in the comments shilling kagi, I will shill my prefered engines. At least my suggestions will bring awareness to free as in freedom projects. I hope to god people paying 10$/month just to not get datacucked by search engines will also learn something and save their money.
SearX/SearXNG is a free and open source, highly customizable, and self-hostable meta search engine. SearX instances act as a middle man, they query other search engines for you, stripping all their spyware ad crap and never having your connection touch their servers. Of course you have to trust the SearX instance host with your query information, but again if you are that paranoid just self host.
I personally trust some foss loving sysadmin that host social services for free out of alturism, who also accepts hosting donations, whos server is located on the other side of the planet, with my query info over Google/Alphabet any day.
Its nice to be able to email and have a human conversation with your search engine provider thats just a knowlegable every day joe who genuinely believes in the project and freely dedicates their resources to it. Consider sending some cash their way to help with upkeep if you like the services they provide, they will probably appreciate and make use of that 10$ better than kagi.
Heres a list of all public searx instances, I personally prefer to use paulgo.io All SearX instances are configured different to index different engines. If one doesn't seem to give good results try a few others.
Did I mention it has bangs like duckduckgo? If you really need google like for maps and buisness info just use !!g in the query
search.marginalia.nu is a completely novel search engine written and hosted by one dude that aims to prioritize indexing lighter websites little to no javascript as these tend to be personal websites and homepages that have poor SEO and the big search engines won't index well. If you remember the internet of the early 2000s and want a nostalgia trip this ones for you. Its also open source and self-hostable
Finally, YaCy is another completely novel search engine that uses peer-to-peer technology to power a big webcrawler which prioritizes indexes based off user queries and feedback. Everyone can download yacy and devote a bit of their computing power to both run their own local instance and help out a collective search engine. Companies can also download yacy and use it to index their private intranets.
They have a public instance available through a web portal. To be upfront, YaCy is not a great search engine for what most people usually want, which is quick and relevant information within the first few clicks. But, it is an interesting use of technology and what a true honest-to-god community-operated search engine looks like untainted by SEO scores or corporate money-making shenanigans.
I hope this has been informative to those who believe theres only a few options to pick from, I know these options are so unknown to most people.
Thank you! So I can use Google but stop it from doing the CAPCHA shit repeatedly because it detects my VPN? It's abuse of the user and I'm tired of it.
Yup! Enjoy :)
I rage quit Google search the other day over that damn captcha
I'm trying Startpage until I get those alternatives down
I'm sure that's what Google wants. You can either give them your data or you can go somewhere else, but hopefully you choose to give them your data because they make it too inconvenient not to.
Abuse of the user 🤣
Have you seen users privacy settings? They want their data to be harvested. Now come here and let me give you some sponsored content.
I use Kagi right now but search.marginalia.nu and YaCy seem really cool. Hell, I might package YaCy and write a module for it for NixOS :^)
I’m trying Kagi now but I’m having mixed feelings. Search results are mixed at best for some pretty commonplace topics (e.g. Starfield quests or breaking news).
Also, the search limit (for the trial and basic plans) stresses me and I find myself second guessing whether I really need to search for something. I like it but I haven’t come across a “wow!” moment that makes me want to abandon DDG, despite the transparency and privacy-focus.
what transparency and "privacy focus" are you talking about?
They haven't released a single line of code and they required you to be logged in, which makes you uniquely identifiable, and if you paid using credit card, then you gave away your personal identifiable information.
uuuuusing it
I don't mind you suggesting these, they're cool projects, but the "coMmEnTs sHiLLinG kAgI" and "mAke UsE of tHat 10$ beTteR thAn KaGi" stuff is so unnecessary. I mean just... why?
I have a bad habit of mixing personal bias into things when I get into a passionate writing fit and it sometimes comes off as pretentious dickery. I never set out to attack kagi users themselves even if I can now see now it did come across and those comments were unneeded. I was being a pretentious jerk with those comments and apologize to all you kagi users for my assholery.
I do think its a waste to spend money on a search engine, and that open source software instance maintainers could probably make use that $ better than another search engine startup. I am being honest with those personal opinions. But its not my place to judge those who decide they are in a well enough financial spot to pay $ for a service that adds precieved value to their life or where they decide to pay it to.
Its fustrating as a FOSS nerd to see so many people shill yet another subscription based service feeding money into another souless company that makes promises of protecting your data and not selling it to ad companies now but has no gaurentee of holding those promises over time. That's how the subscription services get you once they have you, slowly changing promises and creeping in their money making bs but slowly enough to not be too jarring. Maybe I'm just disillusioned with things after being burned so many times. Best of luck to you though I hope it continues to be a valuable service to those willing to pay for it.
Thanks for the thoughtful reply. I'm not a kagi user, but I'll put in a word for paying for services: a search engine requires time, equipment and hosting to run. Sure some kind people may be willing to run it free. But search in my opinion is an essential service. If it's not working, I want there to be someone responsible, and (in the kindest sense) obliged to get it working.
Going further, if there are new developments in the tech or new features to be implemented, I want there to be an incentive for the operator to implement them.
I don't turn to a hobbyist to give me eyeglasses or fix my car. Similarly I don't turn to an advertiser. In some cases I fix my car myself, but when something is too complex or time intensive for me to handle I'm going to pay to ensure I have that essential good or service.
I honestly believe that it's good to pay for services if you find them good. FOSS does need the money but unless you want to wait for the long term, it's not viable as a user.
I'm quite interested in FOSS stuff but I get disappointed by so much FOSS stuff I now just use whatever feels the best to me
Thank you! I'm intrigued by Kagi but it's a lot of money. I've tried SearXNG before it wasn't great for me, I'll try it again.
I hope you find more success with it this time. Like I said not all SearXNG instances are equal paulgo.io was the first to really click with me and give useful results. Some SearXNG instances won't query google or most other engines making their usefulness rather limited. Also the more popular an instance becomes the more likely it will be rate-limited by search engines which isn't the fault of the instance but can be an occasional annoyance for sure. Not perfect solution by any means but I think SearX would be a great fit for lots of people here who just want google results without all the spyware ad bs
Nice choice of lemmy instance, btw. Pubnixes like SDF rule!
Genuine question, what happens to SearX if google pulls the plug on API access or changes the algorithm in a way that makes it worse?
If Kagi got an actual code audit done I would be a lot more on board with it. The audit they do show appears to just be penetration testing, not focused on the code itself but I don't know much about so maybe there is more to it that I don't understand.
I wish it were easier for developers to monetize their projects while leaving them open source. Tutanota is a good example of open source code used in a paid service. With tutanota however it seems like what you pay for is the service, not the software.
I am not the most knowledgeable person on searxng innerworkings so may be wrong, but searxng instances usually use the 'get' and 'post' commands to request+fetch http/https content not an API key. You can get your own api keys/tokens from google and plug them in to searx in the preferences menu if they ever make it API only. There's a lot of IT academic research that relies on google they will most likely never pull API access fullstop but you never you I guess.
There is not much if anything SearXNG instances can do if google changes algorithm. In worst case scenario it can still index other search engines which themselves scrape google like startpage or engines completely independent like duckduckgo, bing, brave, YaCy, ect. Here is a list of all configured engines SearXNG uses by default you can go into preferences at top right of searXNG to configure what engines you want to use among other things.
It just worries me that there isn't really a google competitor if all the alternatives rely on google not screwing up a product. It seems like honest search results are becoming less of a thing they care about.
I'm trying this instance out at the moment and it feels great! Do you know how to get the autocomplete to work on Firefox android? I'm looking for search suggestion API url but can't seem to find it.
My issue with SearXNG is that I cannot natively use it on mobile (iOS). Might be a small issue for most but I need to be able to type into my browser's search bar and it utilize that search engine. Open browser > navigate to search homepage > enter query is a lot slower, especially if I am out and about and need information quickly.
If there is some way to configure this I'd love to hear about it, but Safari on iOS limits you to a handful of search engines. I use DDG today.
I did a quick search and found instructions to what I think you are asking for hopefully this helps. The query string for paulgo.io is "https://paulgo.io/search?q="
I have also heard Xsearch works
searchengine.party also has the query string links for a multitude of different search engines, as well as a comparison of security tests and privacy policies and other functions
xSearch on the AppStore lets you use any search engine with Safari.
Hey just to update I heard XSearch works if that other thing didn't work out
When you need a scalable service for tons of users, federated isn't going to cut it. This is why Apple wants DDG. Point the bajillion crApple lusers at one of your public instances (or even all of them chosen at random each time) and watch it crash and burn overnight. DDG has tons of servers and the infrastructure to hold up while a ton of people search why their luxury device is slowing down every time Apple releases a new one.
Lol Federation is the definition of scalable. Everyone serves their local users -> a miniscule amount of global traffic, everything but auth always stays local.
Universities have been doing it since the beginning of the internet. Email is the biggest example but there are others: eduGAIN and eduroam are the most notable ones coming out of the academic community.
You are confusing a network of distinct servers with a single point of entry that a search engine would need to be. There is no fallback or distribution of search when everything is directed to a single search point, and pointing people to different search sites per search will remove any per-site preferences.
Do people think about what they say any more, or do they see someone who is trying to carefully explain their problem and just go into pure rage and try to disprove them by spewing things that do not make any sense?
No search engine has a "single point of entry". Every search engine has Cache servers all over the world at almost every major IXP. Nothing would prevent a federated service from operating the same way. Cloudflare or literally any form of loadbalancer or load balancing service could be used to redirect queries to fedisearch (or whatever the service name would be) to the local instance by IP geolocation. Authentication can just be forwarded to the home server via SAML, thats also where the settings can be stored and queried at login time by the local instance. SAML assertions are very scalable, and there needs to be no global login server, since every users login query can be forwarded to his home instance, where his profile is loaded. The full search index could be put into a blockchain that every local instance joins - every instance crawls their area and publishes new results to the chain. You seem to know very little about how the internet works, yet you accuse me of raging.
That the foss community can manage things like that has been proven for years. Debian mirror server network works in a similar way (they run their own loadbalancer ofc), while being cryptographically secure. And if you wanna see a federated login network like i described in action, just go to https://pubs.acs.org/action/ssostart
All these parts i described are existing technology and in global use. The combination is not, but there is nothing that would prevent a foundation from implementing search like this.
I don't know what the fuck is going on with Kagi on Lemmy. They must be using bots or paying people for promoting them. I just don't get how people can trust them so much when they haven't released the code for anything, they require you to be logged in which makes the user uniquely identifiable and therefore could easily correlate your searches to your identity (even if they claim not to, it's just a "trust me, bro")
What is a kagi
Yeah it's ridiculous, seems like there's a shill army of this proprietary service here.
Shilling means that you're claiming people (like me) who've been recommending Kagi are in fact secretly paid to do covert advertising.
Are you using that word wrong, or do you actually believe that we're all liars?
Saved. Thank you for your time and thoroughness!
Thanks Yacy looks promising
You're welcome updated the comment with proper links
Would also recommend whoogle. They have done public ones iirc but also hand a self hosted option (that I use behind a VPN) for those that line self hosted shit
thank you, great post
i have tried these alternate search engines and I have to always come back to google. A friend I trust a lot swears that kagi is the best search engine, and so do other people I know, so it must have some merit.