Cloud API Costs 200,000 Yen a Month—Will You Really Keep Paying?

When small and medium-sized enterprises (SMEs) try to utilize AI, they are first confronted with the barrier of “monthly costs.” Integrating ChatGPT API or Claude API into operations can easily cost between 200,000 to 500,000 yen per month. Adding image recognition or multimodal processing can further escalate these costs.

“AI is convenient, but it doesn’t make sense for our scale”—many business owners have likely decided against implementation based on this judgment.

However, that premise is beginning to crumble. The rapid evolution of small models, edge-oriented compression techniques, and the reuse of LoRA (Low-Rank Adaptation) have converged to reveal a path to operating AI in-house without GPUs, without the cloud, and for under 20,000 yen per month.

This is not a “future scenario”; it is already a topic of ongoing research and validation. Let’s explore what has changed.

1. Quantization of Small VLMs—What Can Be Achieved with 300 Million Parameters

The first key point is that the models themselves have dramatically shrunk in size.

Recent research has verified that small VLMs (Vision Language Models) with fewer than 300 million parameters can operate practically on small devices such as NVIDIA Jetson Orin NX and AGX. The Jetson Orin NX retails for about 50,000 to 80,000 yen. Unlike GPU boards that plug into PCs, it is a palm-sized embedded device.

The key lies in the technique of quantization. Quantization is a method of reducing a model’s computational precision from 32 bits to 8 bits or even 4 bits to lighten the model. Naturally, lowering precision leads to reduced performance—this has been the conventional wisdom.

However, this research has revealed that the model’s structure significantly affects its resilience to quantization. Specifically, models with a Mixture of Experts (MoE) structure can be quantized down to INT4 (4-bit integers) with minimal noise impact, maintaining high performance. In contrast, traditional dense structures show significant degradation at INT4.

What this means is that depending on the choice of model, sufficient accuracy can be achieved even with inexpensive devices. There is no need to rent a GPU server for 100,000 yen per month. By purchasing a single device for 50,000 to 80,000 yen, it can run on electricity alone. Converted to monthly costs, including electricity, this amounts to just a few thousand yen.

2. EdgeCompress—Thoroughly Eliminating “Unnecessary Calculations”

The next technology is EdgeCompress, a multidimensional compression framework. Although it is called a framework, what it does is simple yet powerful: it eliminates unnecessary calculations from multiple dimensions simultaneously.

It combines two specific approaches:

Dynamic Image Cropping (DIC) cuts out only the important objects from the input image, completely skipping the calculations for the background. For example, in the manufacturing industry, only the products on a conveyor belt are recognized, discarding the calculations for the factory equipment in the background. This alone can reduce computational load by 30-50% in some cases.

Cooperative Compression (Compound Shrinking) simultaneously optimizes the depth, width, and input resolution of neural networks. Reducing just one of these factors can easily compromise accuracy, but by adjusting all three in tandem, it significantly reduces computational load while minimizing accuracy loss.

As a result of combining these two approaches, CNN-based image recognition models that previously required cloud GPUs can now operate at a practical level on resource-constrained embedded devices.

In the context of SMEs, routine tasks that use images—such as appearance inspections, inventory image counting, and OCR for forms—can be completed solely on local devices without sending images to the cloud. This not only eliminates the pay-per-use costs of cloud APIs but also provides significant security benefits by keeping data in-house.

3. LoRA Recycling—Tuning Costs Become “Almost Zero”

The third technology is the adaptive merging of LoRA (Low-Rank Adaptation), which may have the most significant impact.

LoRA is a lightweight adapter for fine-tuning large models for specific tasks. Instead of retraining the entire model, it adds small parameter clusters to achieve specialized performance for particular operations.

The issue has been that creating LoRA has previously incurred substantial costs. Preparing data, executing training, and evaluating—though small, these processes require expertise and time.

Recently, research has emerged on “reusing” approximately 1,000 LoRA modules published on Hugging Face Hub. This means that businesses can adapt existing LoRA without having to create them from scratch.

Even more interesting is the unexpected finding from the research: the choice of which LoRA to select is actually not that critical. LoRA with randomly initialized parameters can yield similar performance when merged adaptively. In other words, the essence of this method is not to “find an excellent LoRA” but that the merging process itself acts as a form of regularization (preventing overfitting).

This is great news for SMEs. The concern of “not being able to find a LoRA suitable for our operations” becomes unnecessary. Even a randomly chosen LoRA can outperform the base model depending on how it is merged, potentially eliminating the need to hire data scientists for fine-tuning.

Organizing Cost Structures—Breakdown of 20,000 Yen per Month

So, what would the cost structure look like if operating for under 20,000 yen per month? Let’s do a rough calculation:

• Device Cost: Amortizing Jetson Orin NX (about 70,000 yen) over 24 months → Approximately 3,000 yen per month
• Electricity Cost: 15W operation × 24 hours × 30 days → Approximately 300 yen per month
• Storage/Network: Minimal due to local operation → Approximately 1,000 yen per month
• Model/LoRA Acquisition: Using open source → 0 yen
• Maintenance/Update Labor: Assuming 2-3 hours per month → Equivalent to about 5,000-10,000 yen per month

Total: Approximately 10,000 to 15,000 yen per month. Even with some leeway, it stays under 20,000 yen.

Comparing this to cloud API usage, the cost drops from 200,000 yen to 20,000 yen. That’s one-tenth of the original cost. This results in a difference of 2.16 million yen annually. Over three years, that amounts to 6.48 million yen. For SMEs, this difference is equivalent to the cost of a single capital investment.

“So, What Should We Do?”

Now we come to the crux of the matter. Given that it has become technically feasible, what should SMEs do now?

First, take stock of your company’s “image-related operations.” Appearance inspections, form reading, inventory checks, categorizing on-site photos—these tasks are most likely to benefit from small VLMs and edge devices. Understand numerically how much you are paying monthly for cloud APIs or how many hours of labor are spent.

Next, try a small test with Jetson Orin NX or Raspberry Pi 5. There is no need to deploy across all operations right away. Validate whether it “really works” with one operation and one device. Load a small VLM with MoE structure and INT4 quantization, and check accuracy with your own data. This initial investment can be kept under 100,000 yen.

Then, experiment with reusing LoRA. Look for LoRA on Hugging Face that is close to your business operations and try adaptive merging. There is no need to find the perfect LoRA. As research shows, the merging mechanism itself will enhance performance.

The Real Change Is “Who Can Use AI”

Finally, I want to touch on the essence of this change.

The 200,000 yen AI operation cost effectively acted as a filter that allowed only companies with annual sales of several hundred million yen to use AI. With the cost dropping to 20,000 yen, even a small factory with annual sales of 50 million yen or a retail store with ten employees can operate AI in-house.

This is not just a story of “lower costs.” It represents a structural change that widens the base of companies that can utilize AI by more than tenfold.

Large enterprises will continue to run more advanced models in the cloud. However, the AI needed on the ground in SMEs is not cutting-edge massive models. A small model that can distinguish their inspection images, a lightweight OCR that can read their forms, and a simple VLM that can count their inventory photos—these are more than sufficient.

And the era where such “sufficient AI” can be obtained for 20,000 yen per month is already upon us.

The democratization of edge AI is accelerating, including movements from startups like PrismML that focus on model compression for iPhones. No one will implement this for you if you just wait. Start with one device and give it a try.