Conclusion

NVIDIA’s GPU monopoly has begun to erode from three simultaneous directions.

1. NVIDIA itself has open-sourced Run:ai.
2. Amazon has accelerated its transition to its own chips with TorchNeuron.
3. Tenstorrent has raised $6.2 billion (approximately ¥930 billion) to make a push with open-source AI chips.

Let’s break down the significance of these three developments from the perspective of small and medium-sized enterprises (SMEs).

“So, how much will our AI costs actually decrease?”—I will answer this question with a timeline and figures.

—

① NVIDIA Open-Sources Run:ai—A Shift from ‘Lock-In’ to ‘Winning with Efficiency’

NVIDIA has open-sourced Run:ai, which it acquired for about $700 million in 2024, under the Apache 2.0 license.

What is Run:ai? Simply put, it is “software for managing GPU traffic.” It dynamically allocates GPUs to multiple AI workloads, ensuring that idle GPUs are not left unused. In cases reported by companies that have adopted it, GPU utilization has jumped from the traditional 15-30% to 70-90%.

What does this mean?

With the same number of GPUs, the number of AI jobs that can be processed increases by 3 to 6 times.

For SMEs renting GPU instances in the cloud, the direct benefits may seem limited. However, that is not the essence of the matter. If cloud service providers incorporate Run:ai into their infrastructure, they can serve more customers with the same hardware. This efficiency gain creates a structure where the cost is reflected in the pricing.

Another important point is why NVIDIA chose to make this open. The answer is simple: “They can no longer win through lock-in.” As competing chips emerge, opening up the software to broaden the ecosystem ultimately maintains an environment where NVIDIA GPUs are more likely to be chosen—this is a defensive strategy that also raises the overall GPU efficiency in the market.

Impact on SMEs: There will be pressure for cloud GPU costs to decrease by 10-20% over the next 12-18 months. The benefits will come in the form of lower cloud fees rather than direct use of Run:ai.

—

② Amazon’s TorchNeuron—’Running on Non-NVIDIA Chips Becomes the Norm’

TorchNeuron, announced by Amazon, is a development tool that makes it easy to run PyTorch models on AWS’s proprietary chip, Trainium.

The previous problem was clear. It was too cumbersome to transfer AI models optimized for NVIDIA’s GPUs (especially the CUDA ecosystem) to other chips. Code rewriting, re-optimization, and functional verification—engineers spent too much time on these tasks, ultimately having no choice but to stick with NVIDIA.

TorchNeuron dramatically lowers this “migration cost.” If code written in PyTorch can run almost as-is on Trainium, the spell of being locked into “only NVIDIA” will be broken.

Let’s look at the numbers. AWS claims up to 30-40% cost savings compared to equivalent NVIDIA GPU instances for Trainium2. For instance, if inference processing on a p5 instance (equipped with H100) costs ¥500,000 per month, it could drop to ¥300,000-¥350,000 on a Trainium-based instance.

However, there are caveats. Trainium still lags behind NVIDIA GPUs in terms of supported models and optimization maturity. At this point, it cannot be said to “switch everything to Trainium.” But the direction is clear.

Impact on SMEs: By late 2025 to 2026, options will emerge for AWS inference and fine-tuning costs to effectively decrease by 20-40%. Particularly, the reduction in inference costs (the side running AI models) will come first. A monthly API cost of ¥100,000 could drop to ¥60,000-¥80,000.

—

③ Tenstorrent Raises $6.2 Billion—The ‘Third Option’ of Open-Source Chips

The $6.2 billion (approximately ¥930 billion) funding raised by Tenstorrent demonstrates the market’s high expectations for this company led by AMD founder Jim Keller.

Tenstorrent’s approach is fundamentally different from others. It designs chips based on the RISC-V open architecture and also provides IP licenses for the chips. This means they sell blueprints to companies that want to create their own AI chips.

What does this trigger?

The number of AI chip manufacturers will increase. Major South Korean companies like Samsung, Hyundai, and LG have already invested in Tenstorrent. As these companies begin to create AI chips optimized for their own use, the supply of chips will increase, intensifying price competition.

Tenstorrent’s chips are also said to target a price range that is 30-50% cheaper than NVIDIA products. However, this is still a forecast before mass production, and actual prices will become clear after 2026.

The important aspect is the “structure.” NVIDIA’s monopoly has been maintained through the lock-in of the CUDA software ecosystem. Tenstorrent is attempting to circumvent this barrier with the open standard of RISC-V. Coupled with the maturation of the PyTorch ecosystem, a world where “it runs without CUDA” is approaching.

Impact on SMEs: Direct benefits will be seen after 2026. However, the mere existence of Tenstorrent will exert downward pressure on the prices of NVIDIA and AMD. Prices will drop simply due to competition.

—

Organizing the Timeline—When and What Should SMEs Do?

Organizing the three movements along a timeline looks like this.

Now to the End of 2025 (Immediate Actions)

• Start comparing cloud GPUs. In addition to AWS, Google Cloud, and Azure, consider emerging clouds like CoreWeave and Lambda Labs. It is not uncommon to see price differences of 2-3 times for the same GPU.
• Review inference costs. Prices for OpenAI API and Claude API have been dropping at a pace that could halve them in six months. Discard estimates from six months ago.
• The open-sourcing of Run:ai will make it easier to negotiate lower prices for managed GPU services. Simply asking vendors, “Isn’t the efficiency up with Run:ai?” can have an effect.

Early 2026 (Expanding Options)

• AWS Trainium-based services will mature to practical levels. It will be worth actively testing for inference purposes.
• More frameworks will support chips other than NVIDIA. Designing to avoid lock-in to specific chips will become important.

Late 2026 to 2027 (Price Disruption Intensifies)

• Cloud services featuring Tenstorrent chips may emerge. This is where price competition could accelerate significantly.
• A scenario where GPU costs drop to 40-60% of current levels becomes realistic. A world where AI costs of ¥200,000 per month become ¥80,000-¥120,000.

—

Cost Estimation: How Monthly AI Costs of ¥200,000 Will Change

Let’s calculate based on a case where an SME spends ¥200,000 per month on cloud AI.

This estimation assumes usage centered on inference. If large-scale model training is involved, different calculations would apply, but most SMEs will either “use off-the-shelf models” or “fine-tune them,” making this estimation applicable in many cases.

—

What to Really Consider—What Will You Do When Costs Are Halved?

While we have discussed costs up to this point, the essence is not there.

“What can you do that you are not currently doing when costs are halved?”—This is what should be considered first.

When ¥200,000 becomes ¥100,000, will it end with “I saved ¥100,000”? Or will it lead to “I will increase one more application of AI with the saved ¥100,000”?

A decrease in GPU costs means a lower barrier to using AI. AI that was only accessible to large corporations can now be used by companies with just ten employees. This is not just “cost reduction” but rather the “democratization of capability.”

For local SMEs, this change is nothing but a tailwind. The same AI models used by large corporations can now be utilized at less than half the cost. This is because large corporations have heavy investments in existing NVIDIA infrastructure, making transitions slow. More agile SMEs can leap at new options.

There are only three things to do now.

1. Accurately grasp your current AI costs. Know how much you are spending and what you are spending it on. Without this knowledge, you cannot reduce costs.
2. Reassess cloud AI prices every six months. Prices are dropping at a pace where the common knowledge from six months ago no longer applies.
3. Create a list of what you will use when costs drop. It is too late to start thinking about this after costs have decreased.

The collapse of the GPU monopoly is not a story of “just waiting for prices to drop.” Only those companies that are prepared to act when prices fall will benefit from this structural change.

Preparation can start today.