Search engines are useless nowadays

Bakedpotato

I find every site I land on nowadays is sparse on real information and seems like SEO stuff. In the past 15 years it has got progressively worse. I found in the 2010s i could find stuff on the internet and nowadays I have to know where the useful sites are. On Google there's an AI overview that often has wrong information and I noticed Duckduckgo is pushing AI too. When I search I dont want to use AI I'm looking for something written by a human. AI use useful sometimes but I'm not sure I want to use it much.

Johnnyloans

Bakedpotato

For me, google is the worst search engine. I use Duck duck go

KleinesSinchen

There is SEO, there are content farms with automatically generated pseudo-articles (even before LLMs have been widely used). Then we have advertorials in the form of fake comparisons that come to the inevitable conclusion that the solution offered by the company that uploaded the seemingly informative text is the best one.

You can sometimes avoid unwanted summaries by adding -AI to your search.

Generally you are right. Even a search with very precise keywords can lead to a bunch of rubbish results. A classic example of "Enshittification" → SEO and content farms and all that seem to work. As always money is driver here. Gathering visitors, showing ads, making people click on ads.

DeletedUser620

KleinesSinchen Enshittification

Which will continue as long as most people by far accept it.

vtek

Indeed the search engine scene has gotten really bad. I think what's needed is a P2P, distributed, censorship resistant engine, but getting people to adopt it is a problem. There are a couple of projects out there but none seem to be gaining any real traction. Currently i use a meta engine.

Mojeek has good values, respects privacy, is relatively unbiased and is usable, but their index is still quite small. Building a proper search engine, like Mojeek is doing, is extremely expensive which is one reason why the distributed route seems to be the smarter option.

ErnestThornhill

vtek but their index is still quite small.

Compared to what? Google? Its index is over 9 billion as of last year.

oldschkool

@Bakedpotato - with Brave Search users can disable "Answer with AI" and use it on demand only when wanted.

vtek

ErnestThornhill Compared to what?

Compared to the total number of indexable docs on the www, obviously, because any comprehensive search engine needs to index the entire indexable web. If you want to compare to Google, Mojeek's index is roughly 2.25% of Google's _400b pages. At this time Mojeek is ok for very general, popular searches, but for research purposes where one needs to get deep in the weeds, it's largely useless.

I'm not knocking them, but their index is still way too small to be of use for journalists and researchers and the like.

router99

Johnnyloans even DDG has gotten bad with decent search results

guser39

DuckDuckGo uses Bing heavily (or completely?). So, the quality is probably dropping as Microsoft struggles with its own quality control.

Also, DuckDuckGo offers a NoAI option (noai.DuckDuckGo.com). Unfortunately, there is a bug in Chromium/Vanadium where you can't actually set the NoAI DDG as your default search engine. You can add it as a search engine option but it isn't used when you try to search. So, I've set it as my homepage and that mostly works.

scythian2000

What are everyone's thoughts on Kagi?

konem

scythian2000 Works without JS (and look pretty), has onion-mirror, you can turn off AI summary, create/use scopes (“lenses”) for search and block/lower certain domains from search results (e.g. numerous AI slop SEO sites, wikihows and such), advanced search is pretty cool. Costs 100 bucks a year though for unlimited search and there are rough edges (their Privacy Pass for using Kagi in Tor without linking your account to Tor session is not really convenient). And yeah, they are mostly proprietary.

guser39

scythian2000 they are a GenAI company first that also provides search. So, I don't trust them to respect user privacy. It's notable that they require you to pay for GenAI to use their search. They also don't provide Google-free APKs; you have to download the app from the Google Play Store. The lack of an open APK is also a significant tell on the quality and trustworthiness of their service.

I do not recommend Kagi.

zenit

guser39

I'd be happy to try it and possibly pay for it, too, if they offered a plan without AI. Until then, I can only conclude they are an AI search company (or, maybe their non-AI search isn't worth paying for?).

Kagi does offer some AI products because they see them as a complementary search paradigm that many people find useful. While I respectfully disagree and don't use most of these, it is a fact that their Individual plan pricing has not changed since their search came out of closed beta in 2022 (so you are not "paying for AI"). Those that regularly use the Assistant feature with "premium" models and the API are paying for their use over the cost of their subscription, though.

scythian2000
My personal experience is that Kagi's search cumulatively saves me a couple days time each month because I'm not wading through pages of ads posing as results and SEO-gamed junk. Usually, I find what I'm looking for on the first page of results. And you can boost or block websites from appearing in future results, create custom filters ("lenses") for the kind of results you are looking for, reformat URLs based on expressions, make custom "bangs" and shortcuts, and even search for exact phrases ("verbatim") without the search engine just deciding your quotes are negotiable. Put another way, I gladly pay for a subscription so I waste less time using a search engine. Their online documentation is worth a look IMO. As someone else wrote here, they are a search-first company not an AI search company.

scythian2000

konem I agree with all your points. The pricing, in my opinion, is the thing preventing me from renewing my subscription. _€12/month (VAT and currency conversion rates) for the unlimited search plan isn't worth it for me. Especially when DuckDuckGo's results aren't all that worse. And even when using Kagi, I encounter slop sites. It's nice that you can block them, but a new one takes its place sooner than later.

shapirus

Right, traditional internet search results are mostly trash these days. They try to shove down upon you what you didn't actually search for, and searching for exact words (using quoted phrases) doesn't help much. It creates the impression that you're the only person in the world who has encountered this particular problem that you're trying to find a solution for.

I find myself using LLM chats to search and aggregate the information for me increasingly more frequently these days. This seems to work rather well pretty often.

Finding a dedicated resource (forums etc.) related to a certain topic and using its internal search is another good option.

brucugu

guser39 they are a GenAI company first that also provides search. So, I don't trust them to respect user privacy. It's notable that they require you to pay for GenAI to use their search.

that's not true, they are a search first company. They are not making users pay for AI in order to have search, they just include the AI in the plan, of which the expensive part is the search because they do not include a lot of tokens amd they also spend a lot of money on quality crawls.

Divby0

Why not use your own (meta)SE? Something like this e.g.:

https://github.com/searxng/searxng

brucugu

Kagi search results have made me believe again in search engines. Not great, but much more acceptable than ddg.

guser39

brucugu if they're not an AI search company then why is AI built into their plans? You cannot convince me it's not AI search until they remove it from their basic plans. You are paying for AI search, regardless of whether you can "disable" or ignore the assistant.

I am happy that you are enjoying Kagi search.

I'd be happy to try it and possibly pay for it, too, if they offered a plan without AI. Until then, I can only conclude they are an AI search company (or, maybe their non-AI search isn't worth paying for?).

Edit to add:

The Starter plan offers 300 searches per month and 300 AI interactions per month with standard models for USD $5 per month + tax.

I'd rather have 600 (or 1000) searches and zero AI interactions. Why is this not an option?

brucugu

guser39 There is a thread in their feedback website where they are discussing that with the community. I think it is already planned.

Byku

In Google case making search engine more shit was a deliberate process:
https://www.wheresyoured.at/the-men-who-killed-google/