A Monthly Fee of 30,000 Yen, or 360,000 Yen a Year. Will You Still Keep Paying?

Paying 30,000 yen a month for ChatGPT API or Claude API can be a significant fixed cost for small and medium-sized enterprises (SMEs). For a company with five employees, this is even higher than one employee’s smartphone bill.

However, we are now entering an era where AI can be activated from just one USB drive. Zero monthly fees. No need for the cloud. Not even an internet connection is required. This is not just a story of “a little cheaper”; it fundamentally reverses the cost structure itself.

What is happening, how much does it cost, and what changes for SMEs? Let’s break it down concretely.

—

What Is Happening—Three Technologies Have Come Together Simultaneously

There are three key developments to note.

1. The Emergence of USB Bootable LLM Runtime
AI execution environments can now be stored on a USB drive, allowing it to run simply by plugging it in. No installation is required, regardless of the operating system. Tools like “Hermes Desktop” can automatically determine the appropriate model for the hardware and set it up with a single click. We have reached a level where it can be operated even without technical staff.

2. Local LLMs Have Reached a Usable Performance Level
Open-source models like Meta Llama 3.1, Mistral, and Phi-3 are rapidly evolving. Models with 7B to 13B parameters can run on laptops with around 8GB of VRAM. Tasks such as summarizing meeting minutes, drafting emails, and handling internal FAQs can be performed at a quality sufficient for the daily operations of SMEs. In fact, the majority of tasks do not require the performance level of GPT-4.

3. NVIDIA Personal AI Router (PAIR)
This system aggregates the GPU resources of multiple PCs on the internal network to distribute AI requests for processing. No dedicated server is needed. The CEO’s desktop, the accountant’s laptop, and an old gaming PC in the warehouse can all be transformed into part of the “internal AI infrastructure” by utilizing their available GPUs. This means that even if multiple users are accessing it simultaneously, they are less likely to experience delays due to load concentration on a single machine.

With the combination of these three developments, the barrier to implementing “internal AI” has effectively disappeared.

—

Honestly Estimating Costs—Is It Really Zero Yen a Month?

Is the claim of “zero yen a month” overstated? Let’s present the numbers honestly.

Case 1: Starting with Existing PCs Only

This is not an extreme example. If you have a gaming PC purchased after 2020 or a desktop with a GTX 1660 or better GPU, you can start testing today.

Case 2: Purchasing One New PC with a GPU

Three-Year Comparison with Cloud API

In Case 1, there is a difference of over 1,070,000 yen in three years. Even in Case 2, there is a difference of 930,000 yen in three years. For SMEs, this difference is significant. With 930,000 yen, you could hire one part-time employee for six months.

Often Overlooked Costs: Labor Costs

It’s important to be honest here. Local AI does come with the cost of “some effort for setup and operation.” This includes model selection, prompt adjustments, and troubleshooting. Unlike cloud APIs, where everything is handled for you, you will need to manage these tasks yourself.

However, tools like Hermes Desktop and PAIR are rapidly evolving towards being “technician-free.” Even at this point, if you have one IT-savvy employee, it can be set up in half a day. Once set up, daily operations require minimal effort.

—

What Else Changes Beyond Costs—The True Value for SMEs

The impact of zero monthly fees is significant. However, what is truly interesting is the structural change that comes after cost reduction.

1. Data Does Not Leave the Company

As long as you use cloud APIs, internal data is sent to external servers. Customer lists, estimates, internal meeting minutes—everything. Even if you think, “We don’t have any confidential information,” how would you respond if a business partner asked, “You’re not entering customer data into the AI, are you?”

With local AI, data never leaves the company’s PCs. This represents a trust cost for SMEs, protecting their transactions with larger corporations.

2. Resilience Against Network Failures

For SMEs in rural areas, the instability of internet connections is a daily reality. If the internet goes down, cloud AI cannot be used. Local AI can continue to operate as long as the PC with the USB drive is running, regardless of typhoons or network maintenance days.

3. The Barrier to “Testing” Disappears

Cloud APIs are billed based on usage, so there is a psychological cost associated with “let’s try using it.” If employees use it freely, the bill can spike at the end of the month. With local AI, it’s zero yen no matter how many times it’s used. An environment where all employees can freely interact with AI directly correlates to increased AI literacy.

This might be the most significant change. The main reason SMEs struggle to utilize AI is not due to technology or cost, but because “employees are not accustomed to using it.”

4. No Individual Dependency

The entire AI environment is contained within the USB drive. This means that if the same USB is plugged into another PC, the same environment can be replicated. There is no longer a situation where it only works on “that person’s PC.” This makes it resilient against turnover and transfers. Since the system is encapsulated within the USB, handovers can simply be done by “passing this USB.”

—

Caution—Situations Where Local AI May Not Be Suitable

Local AI is not a panacea. Let’s be honest.

• When Cutting-Edge Performance Is Required: If you need inference quality on par with GPT-4o or Claude Sonnet, cloud solutions still hold an advantage. However, it’s worth questioning whether “that quality is truly necessary.”
• High Volume of Simultaneous Access: In scenarios where 50 employees are using it heavily at the same time, the GPU resources of a single PC may not suffice. Even with PAIR, there are limits.
• Advanced Processing of Multimodal (Image/Sound) Data: Local multimodal models are still in development. It’s more practical to start with text-centric tasks.

Conversely, if you have fewer than 10 employees, text-centric tasks, and handle highly confidential data, then local AI is worth considering immediately.

—

So, What Should You Do?

There are just three steps.

1. Today: Buy a USB Drive (1,000 yen)
Download Hermes Desktop and try it on your current PC. It will be operational in 30 minutes.

2. This Week: Conduct an Internal “GPU Inventory”
Check the GPUs in your company’s PCs. You might find that a gaming PC is lying dormant. If you bundle them with PAIR, they can become part of your internal AI infrastructure.

3. This Month: Replace One Task
Summarizing meeting minutes, drafting daily reports, generating templates for inquiry emails—anything will do. Just switch one task from the cloud API to local. If the quality is satisfactory, you can gradually expand.

What’s important is not to “start after fully understanding everything,” but to “test it for 1,000 yen and then think about it.”

—

What Happens When the Cost of AI Approaches Zero?

Finally, let’s discuss a bit about the future.

When the cost of using AI approaches zero, the question of whether or not you are using AI will no longer be a differentiator. Everyone will be able to use it.

At that point, the differentiating factor will be whether you can design what to have AI do. Where in the workflow can AI be integrated for maximum effectiveness? Which data should be fed to create unique value for your company?

This puts SMEs at an advantage over large corporations. Decision-making is quicker. Business flows are shorter. If the CEO says, “Let’s start using this tomorrow,” it can change by tomorrow.

One USB drive. 1,000 yen. Zero monthly fees.

With the cost of tools eliminated, the only question left is “What will you do?”