AI started to affect Google's search
As probably everybody knows, Google created a real monopoly on the internet search. Many people don’t even know about other search engines. To a certain degree, the word “Google” means to search the internet. I think most of us have heard the phrase “Google it”.
But as we know, nothing is forever, and it looks like that monopoly may end soon. But before I dive deeper into what’s changing, let's see the history of how people find information on the internet, how Google took its place, and what problems it had to face.
History
A long time ago, before any search engine existed, when you needed to find specific information on the internet, you had to know the website that had it. For example, if you wanted to buy a Toyota Corolla, then you would need a list of websites that sell them. But how do you find such a website? Well, somebody had to share it with you.
But with time, more websites were added, it was hard to track all of them, and aggregators were born. In the past, websites like AltaVista or Yahoo aggregated and categorized many popular websites. When you need to find some information about the same car, you go to that website, then go to the Cars section, and then go to the Sales subsection, and hopefully, it will have links to a website that sells them.
As you can imagine, searching for anything this way takes a lot of time and is quite a boring process, and honestly, a huge waste of time. And after people found the website they were searching for, they will need a way to remember it. Many people tend to have a special file on the computer or a physical notebook with links to all useful websites they have ever found, because it takes a lot of time to find the website again. And there was also a quite high possibility that you would never be able to find this site again. The bookmark feature that is now present in all browsers today was born at that exact time.
And as you can imagine, there will always be people who are trying to cheat a system and make easy money. As soon as someone started to pay for ads on the internet, they were creating websites that are full of ads, and they are trying very hard to make you visit that website.
But it wasn’t easy because aggregators were using people to validate websites’ content, and they obviously would not add any bogus websites. Bad actors tried to be creative, and they were leaving links to their websites everywhere they could, with the expectation that people would click on them just to find completely unrelated content and a lot of ads. But it wasn’t very successful, because most websites were human-moderated at that time.
But let's go back to search. With the internet boom on the rise, it was clear that things must change because thousands of new websites were added every month, and it wasn’t possible to manually scan and catalog all of these new websites.
And then search engines were born. The search engine crawls all pages of the internet and indexes them. Initially, they only scanned the title and the keyword section of the website and then added that information to the database.
When you type “Buy Toyota Corolla”, the search engine accesses the database and tries to find links that have the words “Buy”, “Toyota”, and “Corolla” and then returns results to the user.
This was a drastic improvement because it saved a lot of time, and people would find the information they were looking for, and did not need to rely only on bookmarks or saved links. But as we all know, bad actors immediately started to use it for their advantage.
They started to create websites with a lot of ads, but with clickbait titles and keywords. I clearly remember a time when many links in the search results pointed to fake websites without any useful information, but with a lot of ads.
This was the first problem, but the second problem was how to properly compare websites. Let’s say you type “Buy Toyota Corolla”, and two websites match this query. Which website should go first, and which one should go second?
Then Google was born. They were quite different from a typical search engine at that time. Normally, when you go to any search engine, you will see a quite heavy page with a lot of ads, and it took a lot of time to load because the internet was very slow back then. The search results page was also quite heavy and contained a lot of ads, too.
In contrast, the Google start page was very simple and contained only a textbox where you can type the search query, and it loaded very fast, even on slow internet. The search results page was also quite clean and also loaded very fast.
The second improvement was that the Google bot ignores keywords and title and crawls the actual content of the web page itself. It invalidated pretty much all bogus sites at that time without relevant content.
The last improvement was in page ranking. Google would rank a page higher if it had more links to it from other websites. So if two websites match the query “Buy Toyota Corolla”, then the websites with more links to them will be displayed first. The thought behind that people would be more likely to leave links to sites they like, and as a result, the more links, the better the website.
All these improvements dramatically improved the quality of the search, and Google quickly became the number one search engine. Then they introduced Google Ads with a bidding system and made ads affordable for many businesses, and this way, they were able to monetize it. Even today, more than 77% profits of Alphabet are coming from Google search.
Problems
As you can imagine, bad actors immediately try to do something about it. They started to create a lot of websites that point to their website to artificially increase the rank of that website. Google tried to battle it, and that battle is still continues to this day.
After a short time, the term “SEO” was born, which stands for “Search Engine Optimization”. These days, SEO experts create thousands of websites and test different strategies to see which one would affect Google and then apply these changes for their paying clients to artificially boost the rank of their websites.
These days, it is so bad that without SEO, your website will have almost no traction. You can create a very good website that adheres to all modern standards and to all Google rules and requirements. You can place unique and very useful content, but your website will still be very low in ranking because you don’t know how search engine works, what is important to them, and what is not.
Moreover, even if you spend a lot of time researching ranking and make necessary adjustments, the effect will often be only temporary because rules are constantly changing. Also, there is a high chance that some of these adjustments will backfire and reduce the rank of your website over time because you will miss the moment when those “optimizations” become bad.
As a result, the internet now contains millions of useless websites filled with the “right” content that contain no useful information to the user, but they all have a very high rank, and they overshadow useful websites that actually contain necessary information that the user is searching for.
This was the first problem, when bad actors were able to manipulate Google to increase the rank of their page, and a lot of websites that just use filler content that is directed to Google and not to humans.
The second problem was greed. For example, if you type “Buy Toyota Corolla”, you will find that the search results page has no real links. On my 30” monitor with 1600 pixels of vertical resolution and 100% scale, the search results page contains 4 sponsored links, and by the end of the 4th link, almost 4/5 of the vertical space is gone.
Then I see the “Places” section with 3 places, and that’s all I can fit on a single page. Then, if I scroll down, I can see the “People also ask” section, and only after that, I can see links to actual websites.
Basically, this is not what I want to find, and instead, I see what Google wants me to find. And it is not because Google does not understand what I want, but because they want to direct my attention to things that make them money.
The last problem with Google is that it works quite badly with complex queries. Logically, you would assume that the more words you provide for the search engine, the fewer results you will get because more words are supposed to narrow your search. But ironically, it works quite the opposite. The more words you provide for Google, the worse results you get. This happened because more words allow Google to match more websites to your query.
For example, if you search for “Go over array in Python on embedded Linux”, you will find the link “Comparing Go vs. C in embedded applications”. That page mentions Python only a few times, just for comparison. Clearly, that page is not what I am looking for.
There will be pages just about embedded Linux that mention that they already have prepackaged Python. There will be links about the Go language on embedded Linux. Sometimes pages I open don’t have one or two words I specified, but Google decided that it is still ok. I think you got the idea of how it works.
Imagine I wrote some code and something does not work as I expected. Clearly, I missed something. I type my query in Google and start opening pages. Unless I made a very trivial mistake, the first pages will not help me, and I will have to open more and more pages.
But new pages often repeated exactly the same advice. They are the same as the previous pages; they were just ranked lower. As a result, I have to read and filter out practically the same advice that I already read on previous pages.
Depending on the problem’s complexity, I have to scan many pages just to find a few hidden gems that I haven’t tried yet. And as you can imagine, it takes a lot of time. Especially if you are on your phone.
And to nitpickers, it has not only happened when you search for something related to IT. I was searching which type of tires my Tesla has. Top pages provided old tyres that Tesla was using that were clearly not the tires my car has. I had to go over 20 pages before I was able to find other options.
And again, it is clear that something must change, because our productivity depends on how quickly we can find the necessary information, and it is clear that the current way does not work well in many cases.
Solution
And then AI comes to the rescue. It can understand the relation between words much better than the Google search engine. So, for my previous query “Go over array in Python on embedded Linux”, AI will understand that I want to have a code in Python that iterates over an array. It will know that I’m not asking about the Go language, and I’m not asking about some information about embedded Linux.
The second good thing about AI is that a more specific query will narrow the scope as it should. So if I change “Python” to “Python version 2”, it will return results for Python version 2 and not random pages that contain the word “version” or the digit “2”. Moreover, for AI, digit will actually mean something because Google most of the time simply ignores digits.
But what’s important is that in any case, you will get a textual answer to your question, and in most cases, there is no need to visit the page that the answer comes from. As a result, it is much harder to monetize clicks, and for bad actors, there is no point in keeping and paying for a website that brings no money. So theoretically, it should reduce the number of bogus sites considerably.
But now we have come to probably the best feature of AI, and that is context. If I have a problem and I ask AI about it, it will provide several solutions. Then I try, let's say 2 of them, and they don’t work. Then AI will try to provide other solutions that I haven’t tried yet. Now I don’t need to filter tens of pages and waste a lot of time on that.
In my case, I started to use AI around a year ago, and I almost stopped searching the internet using any traditional search engine because there is no point. I got an answer to my question immediately and within context. I am only using traditional search to verify information, and that’s it.
And slowly AI starts to hurt business of Google a lot because traditionally, Google got around 77% of its revenue from text search ads. For example, in my case, I reduced searches from tens searches per day to pretty much none.
For example, I need to check how long is Tesla Model Y, Long Range, AWD is. I don’t care about a website that contains this information. All I need is an answer to my question. But Google forces me to check several websites, perhaps requires using the search function of the browser to find this information on the page I opened. But all I need is a simple answer: 188.6 inches or 4.8 meters.
AI considerably shortens the path to getting this information. I got this information in a few seconds with any AI. In a traditional or in a “Google Way”, I will need to open a page with the specifications of that car, and they will use my eyes to find a “Length” column, and then get my answer. In the best case, it will take around 20 seconds.
But if I need some specifics like the length of the model 2020 or 2025, then often Google will not return the correct pages, and it will take at least several minutes to find the necessary information. And if you need even more specific information, for example, the length of that car that is produced in China, then you will need to spend tens of minutes going through the same information on different websites over and over, trying to find out if this information is relevant to you. Most of the time, it will be absolutely relevant.
Conclusion
AI shines because the business of Google is fundamentally wrong. They have a perfect tool that allows their customers to search for information on websites, but people need just information, and most of the time, they don’t care what website it comes from.
As a result, we will see a severe decline in search traffic on Google in the next few years because it is much simpler to get most information using AI, and Google must address this issue.
Is the AI solution perfect? Of course not. You don’t see where this information comes from, and the company that controls AI can adjust the AI in any way they like. Also, AI is typically trained on data from the internet, and we all know that you can find on the internet absolutely anything from flat Earth to lizard people. Lastly, sometimes AI simply hallucinates.
AI is just a tool, and it is up to us how we use it. And obviously, we need to use our brains and not just blindly trust everything AI produces.
I hope it helps somebody.