Conclusion

Let’s get straight to the point. The “price collapse” of AI is happening simultaneously from three directions.

In the same week that Nvidia announced it would invest $500 billion (approximately ¥75 trillion) in AI infrastructure, a consumer GPU priced at ¥150,000 became capable of running a 120 billion parameter AI model.

At first glance, these two pieces of news seem entirely unrelated. However, they share the same essence.

The “cost of using AI” is being dismantled from both the top and the bottom.

This structural change presents the biggest opportunity for local small and medium-sized enterprises (SMEs) in recent years. Let’s explain this step by step.

—

The First Collapse: What Nvidia’s $500 Billion Means

Nvidia’s $500 billion investment, in collaboration with major Wall Street financial institutions, might seem like a story only relevant to large corporations. But that’s not the case.

This money will go towards building data centers and mass-producing GPUs. What happens when mass production occurs? Prices drop.

In fact, the cost of GPU usage in the cloud has plummeted over the past two years. At the beginning of 2023, renting an A100 for an hour cost between $3 and $4, but now there are services available for around $1. Once the $500 billion investment becomes fully operational, this decline will accelerate even further.

In other words, Nvidia’s massive investment is a structural pressure to lower the “usage fees” for AI. This is not just a story for large corporations. The reduction in cloud prices directly impacts the API usage costs for SMEs.

—

The Second Collapse: Running a 120B Model on a ¥150,000 GPU

This is the crux of the matter.

A new quantization technology called “RotaryQuant” has emerged. What it enables is the ability to run a 120 billion parameter Mixture of Experts (MoE) model on a consumer GPU with 16GB of VRAM.

Specifically, the massive Nvidia Nemotron-H 120B model can now perform inference with just 32GB of system memory and 16GB of GPU memory. This means GPUs like the RTX 4060 or RTX 5060, priced at ¥100,000 to ¥150,000.

This is an extraordinary development.

Just six months ago, to run a model of this scale, you would need a server with multiple A100s, costing at least several million yen, with monthly cloud costs in the tens of thousands of yen. Now, it can run on a single ¥150,000 GPU right on your desk.

To simplify the workings of RotaryQuant, it combines mixed-precision quantization, where important parts are kept at high precision while others are at low precision, with offloading techniques that store unused experts (from the MoE network) on disk.

Setting aside the technical details, what’s crucial from a business perspective is the change in cost structure.

The point that “data does not leave the premises” is critically important for local SMEs. There is a strong resistance in the field to uploading customer data, supplier information, and internal know-how to the cloud. If it can run locally, that barrier disappears.

—

The Third Collapse: The Shock of AI Running in Browsers with WebGPU

The third wave is WebGPU.

This technology allows for “direct access to the GPU within the browser,” with major browsers like Chrome, Edge, and Firefox making progress in supporting it. What changes is that AI inference can run on your PC simply by opening a browser, without even needing to install an application.

Compared to the traditional WebGL, WebGPU significantly reduces dispatch overhead (the loss when sending commands to the GPU). For small to medium-sized models, it can run at practical speeds within the browser.

What does this mean?

The “distribution cost of AI tools” becomes zero.

If you create an AI tool for internal use, the traditional setup would require “installing Python on each PC, installing libraries, downloading models…”—a hellish setup. With WebGPU, it can be as simple as “just open this URL.”

For IT personnel in SMEs (often just one person wearing multiple hats), this difference is monumental.

—

So, What Should We Do?

Let’s summarize the three collapses.

1. The cost of using AI in the cloud is decreasing (due to Nvidia’s $500 billion investment increasing supply).
2. Large models can run locally (thanks to RotaryQuant’s quantization technology).
3. AI inference can be done solely in the browser (with the spread of WebGPU).

These three changes are happening simultaneously. In other words, the three major barriers for SMEs—”AI is too expensive to use,” “setting up the environment is difficult,” and “I don’t want to send data outside”—are all beginning to crumble at once.

So, what should local SMEs do right now?

Step 1: Recognize that the cost of “trying it out” has dramatically decreased.

With a ¥150,000 GPU and free open-source models, the environment that used to cost several million yen is now accessible. The barrier to “trying it out” has dropped dramatically. If you haven’t tried it yet, now is the time to start.

Step 2: Experiment with using your own data locally.

If a 120B model can run locally, you can experiment with feeding your own sales reports, customer interaction histories, and manuals to create a “company-specific AI.” Since you don’t have to upload to the cloud, the risk of information leakage is also lower.

For example, a local manufacturing company created a system that can instantly retrieve “causes and countermeasures for symptoms that appeared” by feeding a local LLM with 10 years’ worth of defective product reports (about 3,000 cases). Knowledge that was previously only in the minds of veteran employees is now accessible to anyone. The construction cost was just the ¥150,000 GPU and two weeks of work from an in-house engineer.

Step 3: Create “distributable AI tools” using WebGPU.

With WebGPU, you can distribute AI tools to clients and partners with just “one URL.” For example, automatic quote generation, natural language search for product specifications, and simple chatbots. No installation required, zero server maintenance costs. This is entirely feasible even within the limited IT budgets of SMEs.

—

The Real Competition is Not “Do You Have AI?” but “Do You Have Your Own Data?”

This is the most important perspective.

AI models themselves are open-source and free. The hardware to run them costs ¥150,000. Cloud usage fees continue to decrease. In other words, the “tool value of AI” is rapidly becoming commoditized.

So, what will be the differentiating factor? Data.

Large corporations have vast amounts of data, but it is often “broad and shallow.” In contrast, the data held by local SMEs is “narrow and deep.” It consists of dense data that is unique to specific industries, regions, and customer segments.

Repair histories accumulated over 20 years. Crop growth data tailored to the local climate. Processing know-how for specific materials. These are not easily obtained by large corporations, no matter how much money they throw at it.

In a world where the cost of using AI approaches near-zero, “what data you possess” becomes the only competitive advantage.

Conversely, SMEs that consciously start accumulating data now will find themselves in an overwhelmingly advantageous position three years from now.

—

Summary: The Walls Have Fallen. The Only Question is “When Will You Move?”

Nvidia’s $500 billion will push down the costs of AI infrastructure from above. RotaryQuant will break through the barriers of local execution from below. WebGPU will dismantle the barriers of distribution and implementation from the side.

Walls are collapsing simultaneously from three directions.

“It’s still too early for us to have AI”—the expiration date on that excuse is rapidly approaching.

With just ¥150,000 and a single URL, that’s all you need to get started.