“The Cost of Being Found Through Search Has Quietly Increased Tenfold”

What was once virtually free has suddenly become costly.

Such a change is currently happening in the act of “being found through search.” Reddit has shut out RSS and APIs, Google’s AI responses are siphoning off search traffic, and AI bots are ravaging the web. As these three changes occur simultaneously, the cost for small and medium enterprises (SMEs) to “be found online” is skyrocketing to 5 to 10 times what it was just 2 to 3 years ago.

This is not just a story about large corporations. For local construction companies, town tax offices, and manufacturing firms with 10 employees—what is happening, and what should they do? Let’s take a closer look.

—

1. Reddit’s RSS and API Restrictions—The “Free Information Source” Has Disappeared

In 2023, Reddit made API usage paid, effectively shutting out most third-party apps. The cost for API usage is approximately $12,000 for every 10 million requests (about 1.8 million yen). What was previously free has suddenly turned into a monthly cost of several hundred thousand yen overnight.

Restrictions on RSS feeds have also been tightened, making it difficult to automatically retrieve data from external tools.

You might think, “Reddit doesn’t concern us.” However, the essence of the problem is not Reddit itself. The fact that platforms can close off what they had allowed to be used for free at any time.

Until now, many SMEs have built their marketing on “free infrastructure” such as free social media posts, organic Google searches, and information gathering using RSS and APIs. Reddit’s actions highlight that this infrastructure has always been susceptible to being dismantled at any moment.

X (formerly Twitter) has also moved to a paid API model, with a basic plan costing $100 per month (about 15,000 yen) and a pro plan costing $5,000 per month (about 750,000 yen). The era of freely disseminating and collecting information is clearly coming to an end.

—

2. Google’s AI Responses—Search Results That Aren’t Clicked

Even more serious is the structural change in Google search.

Google has started displaying AI-generated answers at the top of search results under “AI Overview.” Users can get answers on the spot, eliminating the need to click on links to visit sites.

According to data from research firms Datos/Semrush, there have been reports of organic search click-through rates (CTR) dropping by as much as 60% for queries where AI Overview is displayed. This means that even if you rank first in SEO, you may receive less than half the traffic you used to.

So, is Google properly compensating the publishers of the content used for AI responses?

The answer is “almost none.”

Google has entered into licensing agreements for AI training data with some major media outlets, but the amounts are reported to be in the millions of dollars annually. While this may sound substantial, Google’s advertising revenue from search is around $175 billion per year (about 26 trillion yen). The return rate to publishers is less than 0.01% of advertising revenue.

This structure is clear. Google generates AI responses using publishers’ content, does not return traffic to publishers, and pays almost nothing in compensation.

If this is the situation even for major media, it is becoming increasingly common for blog posts and FAQ pages from SMEs to be absorbed by AI and end up resulting in “zero clicks.”

The Impact in Numbers

Let’s consider a hypothetical SME with a website that receives 10,000 page views per month.

• Two years ago: SEO costs 50,000 yen per month → 10,000 page views per month → 20 inquiries per month
• Now: SEO costs 50,000 yen per month → 4,000 page views per month (60% reduction in CTR due to AI Overview) → 8 inquiries per month

To maintain the same number of inquiries, listing ads need to be used to compensate. If the advertising cost per inquiry is between 3,000 and 5,000 yen, that adds an additional cost of 36,000 to 60,000 yen for 12 inquiries per month. This brings the total to over 100,000 yen per month.

What used to cost 50,000 yen for SEO alone has now ballooned to 100,000 to 150,000 yen per month. This is the breakdown of the “cost of being found increasing tenfold.” In some industries, it could be even more severe.

—

3. AI Bots Scraping—Servers Being Devoured

The third issue is web scraping by AI bots.

OpenAI, Anthropic, Google, and countless other AI companies are crawling websites to collect training data. According to Cloudflare’s 2024 report, approximately 38% of overall web traffic comes from bots, with the proportion of AI-related bots rapidly increasing.

For SMEs, this is a blow for two reasons.

The first is the increase in server costs. If a site is operated on a shared server costing a few thousand yen per month, heavy access from AI bots can slow down the server. In the worst-case scenario, human users may be unable to access it. Implementing CDN or WAF (Web Application Firewall) to counter bots incurs additional costs of several thousand to tens of thousands of yen per month.

The second is unauthorized use of content. Specialized content written on one’s own site can be incorporated into AI training data and restructured as AI responses. Users may be satisfied with the AI’s answers and not visit the original site. Even if you block it with robots.txt, not all bots will comply.

As a result, the more content you create, the more it gets absorbed by AI, leading to a decrease in traffic to your own site. This reverse phenomenon is occurring.

—

So, What Should SMEs Do?

This is not a story about “SEO dying” or “search coming to an end.” Search is still functioning. However, the cost of “relying solely on search for customer acquisition” has structurally increased.

Here are three concrete actions you can take right now.

① Establish Direct Channels

Create “directly connectable channels” that do not depend on platforms, such as email lists, official LINE accounts, and proprietary apps.

The official LINE account allows you to send up to 200 messages per month on the free plan. Even on the paid plan, it starts at 5,000 yen per month. This is a channel you can control, unaffected by changes in Google search algorithms.

Email marketing tools like MailerLite offer free plans for up to 500 contacts, with monthly fees starting in the 1,000 yen range. This should be considered as a “main line” rather than just “insurance.”

② Create “Content That Cannot Be Absorbed by AI”

AI finds it easy to reconstruct general information—articles like “What is XX” or “How to do XX.”

Conversely, what AI cannot replicate is “primary information that can only be generated in your own field.” Case studies with before-and-after images, customer testimonials, and know-how that takes into account local specifics. Even if these are absorbed by AI, they cannot be reproduced in the context of “I want to rely on this company.”

Shift to content that accumulates “trust” rather than just “information.” This is something only SMEs can do. Large corporations do not have the on-the-ground experience.

③ Understand Cost Changes Through Numbers

Don’t just conclude with “I feel like access has decreased.” Check the monthly trends in click-through rates using Google Search Console. Calculate the cost per acquisition (CPA) for each inquiry.

For instance, if your CPA has doubled compared to six months ago, that reflects a structural change in the market, not a lack of effort on your part. By looking at the numbers, your strategies can change.

—

Conclusion: Let Go of the Assumption That You Can Be Found for Free

The monetization of Reddit’s API, Google’s AI Overview, and AI bot scraping—these three may seem like separate events, but they actually point to the same structure.

“The infrastructure that platforms have provided for free is being monetized, degraded, or disappearing.”

Until now, SMEs have attracted customers by placing their products on the “free shelf” of Google search results. The cost of using that shelf is now skyrocketing.

This is not a reason for pessimism. Rather, now is the time for SMEs to leverage their “ability to connect directly” to turn the tables, as large corporations are feeling the same pain. Large corporations are too big to respond in a hands-on manner, such as replying to each individual on their official LINE account. It is SMEs that can digitally recreate the trust relationships rooted in local communities.

The shift from “being found through search” to “going for direct connections”—will you panic after the cost has increased tenfold, or will you prepare in advance? The difference will be made there.