Is Google Search Dying? A Deep Dive Comparing Search Engine Results in 2024

A recent experiment has cast a critical eye on the state of modern search engines, questioning whether the quality of results has declined significantly in recent years. The investigation compared the performance of Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT across a range of queries, exposing a complex picture of strengths, weaknesses, and pervasive issues.

The test involved several categories of searches, including everyday tasks like setting up a new computer, finding answers to common knowledge questions, and local information queries. The goal was to assess the relevance, accuracy, and safety of the results, paying particular attention to the prevalence of scams, misleading advertisements, and low-quality content.

One key finding was the significant presence of "content farms" and SEO-optimized websites that prioritize ranking highly in search results over providing genuinely useful information. This often meant that users struggled to find legitimate open-source projects or reliable sources, buried beneath a deluge of irrelevant or even harmful content. The experiment specifically cited the difficulty of finding `yt-dlp` using the query "youtube downloader" due to the overwhelming presence of these SEO driven sites.

The experiment also highlighted the increasing prominence of recent YouTube videos in search results, even when those videos were of questionable quality or relevance. This phenomenon appears to be driven by Google's algorithm favoring recent content, leading to a situation where low-effort videos from content creators seeking ad revenue can outrank more informative and authoritative sources. An example cited a user searching for an ad blocker and getting a video of someone rambling about ad blockers then searching for an ad blocker themselves in Google and clicking the first link which was an ad.

ChatGPT demonstrated a mixed performance, excelling in some areas but also exhibiting its well-known tendency to "hallucinate" or generate nonsensical answers. However, the study argues that traditional search engines also return various kinds of fabricated results, some being deliberate attempts to trick the user into ad clicks or signing up for fraudulent services.

Marginalia performed relatively well, providing decent answers when able and avoiding answers or providing irrelevant ones when unable. It also had a low rate of scams. Mwmbl allows users to directly edit search result rankings, however the experiment did not account for these edits.

The report concludes that while concerns about AI hallucinations are valid, the traditional search ecosystem suffers from its own set of problems, driven by the incentives to prioritize profit over user experience. This creates a situation where misleading advertisements, biased reviews, and outright scams can thrive, making it increasingly difficult for users to find trustworthy and accurate information.