AI Compute: ‘Bought but Not Used’ — The Reality of Rising Server Costs Driven by FOMO and the Sufficiency of ¥50,000 a Month for SMEs

While large corporations are pouring billions of yen into AI and are “still considering its applications,” local SMEs are increasing their sales by utilizing AI through APIs for just ¥50,000 a month.

Doesn’t this structure seem strange?

—

60% of Secured GPUs Are ‘Idle’ — An Abnormal Situation

Currently, there is an anomaly in the AI investments of large corporations.

According to reports from research firms, about 60% of the AI compute resources secured by companies are unused. In other words, while they contract expensive GPU servers, secure racks in data centers, and pay monthly costs ranging from several million to tens of millions of yen, more than half of these resources are simply “consuming electricity.”

Why is this happening? The answer is simple: FOMO (Fear of Missing Out).

“Competitors are investing in AI,” “GPUs might become scarce,” “If we don’t secure resources now, it will be too late” — this anxiety leads to securing resources before their applications are determined. In management meetings, the amount invested in AI is reported, but the “AI utilization rate” is not. The act of investing itself has become the goal.

This mirrors the previous cloud migration boom. How many companies migrated to the cloud with the mindset of “let’s just move to the cloud” only to find their monthly costs ballooning compared to the on-premise era? They did not learn.

—

9GW in Utah — A Single Data Center Consumes Twice the Power of the Entire State

The waste generated by FOMO is not limited to individual companies; it extends to distortions in social infrastructure.

In Utah, the construction of an AI data center campus with a capacity of 9GW has been approved. How much is 9GW? It is more than twice the total power consumption of the entire state of Utah, equivalent to nine nuclear power plants.

One facility consumes enough power to support the entire infrastructure for 3 million residents of the state twice over. This reality is being justified as “infrastructure investment for the AI era.”

Of course, large-scale model training requires vast computational resources. However, the problem lies in the ambiguity of “who” and “for what purpose” these computational resources will be used, while the facilities are being built in advance. Power is finite. That finite resource is being reserved for an AI whose utility is uncertain.

When discussions arise about attracting data centers to rural areas, it is often welcomed with promises of “job creation” and “increased tax revenue,” but if competition for power arises, the costs for local manufacturing and agriculture will rise. Few municipalities are considering this side effect.

—

Nvidia Executive: ‘AI Costs More Than Labor’ — The True Meaning of This Statement

An Nvidia executive made headlines by stating, “AI costs more than labor.” This comes from an executive of a company that has achieved record profits from GPU sales, adding a layer of irony.

However, this statement needs to be accurately interpreted.

“If you build your own large-scale AI infrastructure, it will cost more than labor” — this is the essence. Building a GPU cluster in-house, setting up a learning environment, hiring an operations team, and paying for power and cooling costs — this full-stack investment in AI infrastructure is indeed more expensive than hiring people.

So, what about SMEs?

There is no need to have their own infrastructure.

With OpenAI’s API, Claude, and Gemini, SMEs can access GPT-4-level inference capabilities for just a few tens of thousands of yen a month. Tasks that do not require in-house training — such as handling inquiries, generating meeting minutes, organizing data, drafting sales emails, and searching manuals — can be sufficiently managed by simply calling an API.

In other words, Nvidia’s statement serves as a warning for large corporations, while for SMEs, it is rather a tailwind. SMEs can access equivalent “inference capabilities” for ¥50,000 a month, which large corporations spend hundreds of millions of yen to build. This asymmetry is the greatest structural change currently occurring.

—

What Can You Do with ¥50,000 a Month? — The Practical AI Utilization for SMEs

When people hear “¥50,000 a month,” they might not grasp what can be achieved. Let’s take a closer look.

• In the case of OpenAI API: Using ¥50,000 worth of GPT-4o allows for processing approximately 2.5 million tokens (about 1.5 million characters in Japanese). This equates to about 50,000 characters per day, easily covering inquiry responses and document creation for ten employees.
• Automatic generation of meeting minutes: Transcribing one hour of meeting audio using the Whisper API and summarizing it with GPT-4 costs about ¥50 to ¥100 per session. Even doing this 100 times a month would cost less than ¥10,000.
• Internal knowledge search: By integrating RAG (Retrieval-Augmented Generation), a system can be created to search internal manuals and past proposals in natural language. Even including the cost of a vector database, this would only be a few thousand yen a month.

This cannot be called an “AI environment on par with large corporations.” However, it is more than sufficient to enhance productivity on the ground.

While large corporations debate “what can be done with AI” in conference rooms, SMEs can “get today’s work done with AI.” This speed difference is actually the greatest competitive advantage.

—

From ‘Ownership’ to ‘Utilization’ — A Structure Where SMEs Can Win

Once, IT infrastructure was something to “own.” Companies built server rooms, purchased hardware, and hired engineers. Capital strength directly correlated with the quality of information systems.

The cloud shattered that. With the advent of AWS and GCP, the shift was made from “owning” to “using.” Even SMEs can now use the same infrastructure as large corporations on a pay-as-you-go basis.

The same thing is happening with AI compute. Moreover, this time it is even more extreme.

Large corporations cannot escape the shackles of “ownership.” Having passed the budget for GPU investments worth billions of yen, they must produce “results” even if they do not use them. Organizational inertia dulls their judgment.

SMEs, on the other hand, lack that inertia. They can stop next month. They can start this month. There is no need for stamps or approvals for API contracts.

The “pay for what you use” model is not just about cash flow. It is about the speed of decision-making itself. Try it out, and if it doesn’t work, stop. If it goes well, expand. Only SMEs can cycle through this process on a monthly basis.

—

So, What Should Be Done?

There are three things SMEs should do.

1. Do not own AI infrastructure

There is no need to buy GPU servers. APIs are sufficient. Whether it’s OpenAI, Anthropic, or Google — it doesn’t matter. Start with ¥50,000 a month, and if it proves effective, scale up. That’s all.

2. Decide on ‘what to use it for’ in advance

The failure of large corporations’ FOMO investments lies in “securing resources first and thinking about their applications later.” SMEs should do the opposite. “Automate the 20 hours spent each month on meeting minutes,” “delegate initial responses to inquiries to AI” — identify the challenges first, then consider the technology.

3. Start with tasks that are dependent on specific individuals

Tasks that can only be done by certain individuals should be the top priority for systematization with AI. Feed the tacit knowledge of veteran employees into RAG to make it searchable. Train AI on sales pitch patterns to aid in onboarding new hires. Eliminating dependency on specific individuals is the most practical entry point for AI utilization.

—

Don’t Get Caught Up in FOMO. Address the Challenges on the Ground.

The mountains of GPUs amassed by large corporations due to FOMO are likely to be looked back on in a few years with the question, “What was that investment for?” Whether the 9GW data center in Utah will truly be fully operational in the future remains uncertain for anyone.

However, there are “challenges that certainly exist today” in the field of SMEs. The era where these challenges can be solved with just one API has already arrived.

With the ability to start for ¥50,000 a month, what reason is there not to begin?

Whether to get swept up in FOMO and waste hundreds of millions of yen, or to confront the challenges on the ground and achieve results for ¥50,000 a month — the smarter choice is clear.

There is no need to mimic large corporations. SMEs have their own way of fighting.