Title

AI Investments by Large Corporations: Half Are Going to Waste—How Can SMEs Turn This to Their Advantage?

Body

Large corporations are pouring hundreds of millions of yen into AI infrastructure. However, half of that investment is going to waste.

This is not an exaggeration. It is the conclusion of a survey conducted by VentureBeat involving 107 companies. About half of these companies have a GPU utilization rate of less than 50%. Less than half are rigorously tracking costs. Furthermore, 57% of companies admit that their AI agents provide incorrect answers.

Investing in infrastructure, failing to measure costs, and lacking trust—this is the “triple whiff” of corporate AI investment.

So, what does this mean for small and medium-sized enterprises (SMEs)? To put it simply, this is an opportunity.

Whiff 1: GPU Utilization Below 50%—”Having” Is Not Justice

Large corporations are rushing to build their own dedicated GPU clusters. They buy hundreds of NVIDIA H100s, expand data centers, and hire dedicated teams. They line up GPUs worth hundreds of thousands of yen each, yet their utilization rate is below 50%. This means that for more than half the time, expensive hardware is sitting idle.

Why does this happen? The management decision to “do AI” comes first, while “what to use it for” is pushed to the back burner. The mere possession of infrastructure has become the goal.

SMEs do not need to fall into this trap. They do not need to own GPUs in the first place.

Now, by using OpenAI’s API, they can access GPT-4-class models for just a few yen per request. By utilizing pay-as-you-go AI services from Google Cloud or AWS, they can obtain sufficient processing power for as little as 50,000 yen per month. Just as companies transitioned from owning their own servers to using the cloud, a similar structural change is occurring in AI computing resources.

What is the difference in output quality between the infrastructure built at the cost of hundreds of millions of yen by large corporations and the API services used by SMEs for 50,000 yen a month? Honestly, in many use cases, there is hardly any difference.

“Not owning” becomes the greatest weapon for SMEs.

Whiff 2: Not Measuring Costs—Thus, “Effects Are Unknown”

Another issue highlighted by the survey is that many large corporations cannot rigorously track AI-related costs.

This is no laughing matter. After investing hundreds of millions of yen, they cannot answer, “So, how much did we spend, and how much did we recover?” They introduce AI tools in a fragmented manner across departments, making it impossible to grasp the overall costs. They run dozens of Proofs of Concept (PoCs) but cannot track which ones have transitioned to production.

SMEs have a significant advantage here. Why? Because they are smaller, they can measure.

For example, if an AI chatbot is implemented for customer inquiries, and the monthly service costs 30,000 yen, reducing the number of inquiries from 200 to 50 means a savings of 225,000 yen if the cost per inquiry is 1,500 yen in labor costs. The return on investment is clear: spending 30,000 yen results in a savings of 225,000 yen. There is no clearer number than that.

In large corporations, this basic act of “measuring” becomes impossible due to organizational complexity. SMEs can cycle through the process of implementation, measurement, and improvement with just one person in charge. This agility translates into a clear competitive advantage when put into numerical terms.

The key is to decide on “what to measure” before implementation. Response time, number of cases processed, error rate, customer satisfaction—any one of these will do. If a Before/After comparison can be made, the investment decision becomes significantly clearer.

Whiff 3: 57% of AI Agent Responses Are Incorrect—Trust Is Built Through Systems

The third whiff may be the most serious. 57% of companies report that their AI agents provide incorrect answers.

This is not simply a matter of “AI accuracy being low.” The essence of the problem lies in the poor quality of the data being fed to the AI. The survey refers to this as the “AI context gap.” The data that AI agents reference is outdated, incomplete, or contradictory. Hence, the answers are wrong.

In large corporations, databases are fragmented by department, and formats vary widely. Customer information may conflict between CRM and core systems. In such a state, there is no way for AI agents to produce accurate answers.

SMEs are structurally advantageous here as well.

The total amount of data is smaller. However, because it is smaller, it can be organized. A customer list of 500 entries, a product master of 200 entries, and 100 FAQs. At this scale, data can be organized in just a week. When AI is run on well-organized data, accuracy dramatically improves.

In fact, in a local manufacturing company we supported, organizing three years’ worth of inquiry data (about 800 entries) and implementing a Retrieval-Augmented Generation (RAG) system led to a 92% accuracy rate in internal inquiry responses. It took two weeks for data organization and one week for system construction. The only cost was the monthly fee for a cloud service, around 20,000 yen.

While large corporations spend a year and several million yen on data integration projects, SMEs can create functioning systems in three weeks.

Another important aspect is the human verification system. Large corporations fall into a double trap by trying to automate the evaluation of AI agents, leading to low reliability in the automated evaluations themselves. For SMEs, it is sufficient to include a flow where a responsible person visually checks the AI’s responses. Spending just 30 minutes a week reviewing AI response logs allows for corrections of any odd answers. This hands-on cycle ultimately produces the most reliable operations.

Three Things SMEs Should Start Doing Today

As they observe the whiffs of large corporations, will SMEs think, “This has nothing to do with us” or will they act, thinking, “This is an opportunity”? This is where the difference lies.

1. Do not own. Start with APIs and the cloud.
There are countless AI services that can be started for less than 50,000 yen a month. OpenAI API, Google Gemini API, Claude API. Focus on trying one specific task first. There is no need to think about company-wide implementation.

2. Decide on one “measurement metric” before implementation.
It can be response time, number of cases processed, or error rate. Have one number that allows for Before/After comparison. Just that will fundamentally change the accuracy of investment decisions.

3. Keep data small and well-organized.
FAQs, customer lists, product information. Start with just 100 entries. Prepare accurate, up-to-date, and uniformly formatted data. The accuracy of AI is determined more by the quality of the data than by the performance of the model.

The Structure Has Changed. What Is Being Asked Is Not “Scale” but “Speed”

The essence of what is happening in the world of AI is the democratization of costs.

Computing resources that were once only available to large corporations are now accessible to anyone via APIs. Machine learning models that could only be handled by experts can now be operated simply by writing prompts.

In this structural change, the advantage of large corporations has flipped from “financial power” to “organizational complexity.” They have the money to invest on a large scale. However, because they are large-scale, they cannot measure costs. Because the organization is large, data cannot be integrated. Decision-making is slow, so experiments cannot be conducted.

The weapons of SMEs are being small, fast, and measurable. This is an unprecedentedly powerful advantage in the AI era.

While large corporations leave GPUs idle, SMEs can achieve results for 50,000 yen a month. While large corporations take a year for data integration, SMEs can create functioning systems in three weeks. While large corporations debate how to evaluate AI agents, SMEs can use, fix, and iterate in the field.

This speed difference is the structure of SMEs’ reversal.

I want to ask: In your company, how much are you spending on AI each month, and what has changed? Can you answer that? If you can, you are already ahead of half of the large corporations.