Conclusion First

Annual cost of 600,000 yen drops to 600 yen—however, making a decision based solely on this could lead to painful consequences.

It is now possible to fine-tune a 3 billion parameter LLM on a smartphone. This is no longer a story of the future. Reports indicate that devices like the iPhone 17 Pro can tune a 3B parameter LLM for each user with just one battery charge.

What does this mean?

Small and medium enterprises (SMEs) that have been paying 50,000 yen per month for cloud AI now have the potential to run AI with just the cost of electricity. An annual cost of 600,000 yen could become just 600 yen. This represents a 99.9% reduction in costs.

However, if you jump at this number without further consideration, you might make a poor judgment. Is it really advantageous? Where are the pitfalls? Let’s calmly calculate the break-even point from the perspective of SMEs.

What Are You Paying for the Cloud AI Subscription of 50,000 Yen?

First, let’s break down the 50,000 yen monthly fee for cloud AI services. A typical scenario for many SMEs looks like this:

• Meeting Minutes Summarization: 3 meetings a week × 4 weeks a month = 12 times a month
• Automating Inquiry Responses: Processing 200 to 500 emails/chats a month
• Inventory Analysis and Demand Forecasting: Daily batch processing

Processing these tasks using a GPT-4 class API would typically cost around 30,000 to 80,000 yen per month, depending on token consumption. A SaaS package priced at 50,000 yen per month is realistic. That amounts to 600,000 yen annually, or 1.8 million yen over three years.

This amount includes model performance maintenance and updates, infrastructure management, security patches, and support. In other words, it’s the cost of “not having to do anything yourself.”

Is the “Electricity Cost Only” for Local Execution True?

Next, let’s calculate the costs of running an LLM on a smartphone.

The energy consumption for fine-tuning a 3B parameter LLM is about 0.5 kWh per session. Assuming the electricity cost is 25 yen per kWh, that’s 12.5 yen per session. Even if you fine-tune it four times a month, that totals 50 yen a month, or 600 yen annually.

The cost of inference (the actual processing when using AI) is even lower. Inference for a 3B model requires only a fraction of the energy used for fine-tuning. Even if you run meeting minute summarization five times a day and inquiry processing twenty times a day, the electricity cost would still only be in the tens of yen per month.

Thus, looking solely at running costs, the claim of “600,000 yen annually → a few thousand yen annually” is not an exaggeration.

However, there are costs that must not be overlooked here.

Uncovering Hidden Costs

1. Initial Investment

An iPhone 17 Pro is estimated to cost between 150,000 to 200,000 yen. However, this is not purchased solely as an “AI-specific device”; if used as a smartphone for daily operations, the additional cost is nearly zero. If employees already have iPhones, they can simply switch at the time of upgrading.

For Android devices, models equipped with Snapdragon 8 Elite are available in the 100,000 to 150,000 yen range, offering AI processing performance comparable to the iPhone.

2. Labor Costs for Setup and Operation

This is the biggest pitfall. Cloud AI services are “set it and forget it,” but local LLMs require you to select the model, prepare fine-tuning data, adjust prompts, and verify output quality.

Do you have personnel capable of doing this in-house? If not, you’ll need to outsource or incur learning costs. Assuming it takes 10 hours of operational labor per month at an hourly rate of 3,000 yen, that amounts to 30,000 yen a month, or 360,000 yen annually.

In this case, compared to the cloud cost of 600,000 yen, you would only save about 40% with 360,000 yen plus a few thousand yen for electricity. The dream of a 99.9% reduction disappears.

3. Differences in Model Performance

This is the point that should be viewed most rationally. There is a clear performance difference between a 3B parameter local LLM and large-scale cloud models like GPT-4o or Claude 4.

• Sufficient Tasks for a 3B Model: Routine meeting minute summarization, FAQ responses, template-based text generation, simple classification tasks
• Challenging Tasks for a 3B Model: Complex analyses involving intricate reasoning, structuring long texts, providing expert knowledge responses, multi-step decision support

About 70-80% of SME tasks fall into the first category. This is a crucial point.

Where is the Break-even Point?

Let’s calculate three scenarios.

Scenario A: When there are AI-savvy personnel in-house

• Local operational cost: Electricity cost of 5,000 yen annually + labor cost of 120,000 yen annually (5 hours a month × 2,000 yen) = approximately 125,000 yen
• Cloud AI: 600,000 yen annually
• Annual savings of 475,000 yen. Profitable from the first year.

Scenario B: When outsourcing operations

• Local operational cost: Electricity cost of 5,000 yen annually + outsourcing cost of 480,000 yen annually (40,000 yen a month) = approximately 485,000 yen
• Cloud AI: 600,000 yen annually
• Annual savings of 115,000 yen. Some savings, but the impact is small.

Scenario C: A Hybrid of Cloud and Local

• Routine tasks (70% of total) handled locally, complex processing (30%) outsourced to the cloud
• Local operational cost: 125,000 yen annually + cloud (30% usage): 180,000 yen annually = approximately 305,000 yen
• Full cloud: 600,000 yen annually
• Annual savings of 295,000 yen. This is realistically the optimal solution.

The “True Value” for SMEs is Not Just Cost Reduction

While we’ve discussed costs so far, the most important aspects lie beyond mere expenses.

1. Data Remains In-House

Using cloud AI means sending meeting minutes, customer inquiries, and inventory data to external servers. With local execution, data remains contained within the device.

This becomes a decisive difference for industries such as healthcare, nursing, legal professions, and manufacturing where “data should not leave the premises.” Companies that have hesitated to adopt AI until now may take the plunge with local execution. A new market will emerge here.

2. Operates Offline

Many SMEs work in environments where internet connectivity is unstable, such as factory floors, construction sites, mountainous farms, or sales vehicles on the move. With local LLMs, AI can be used even in areas without signal.

Whether AI can be used on-site is a watershed moment for SMEs. AI designed for cloud use is tailored for office workers. AI that serves those working in the field, with dirty hands, can only be realized locally.

3. A System That Does Not Become Dependent on Individuals is Embedded in the Device

If you fine-tune the model to remember the judgment criteria of veteran employees, that knowledge remains within the device. Even if people leave, the model remains.

The biggest challenge for SMEs, “dependency on individuals,” can be addressed by LLMs on smartphones. Moreover, there’s no risk of “service termination” like with cloud services. The model is in your hands.

So, What Should You Do?

Here are three points:

1. Try it first.
Run a local LLM on your smartphone right now. Apps based on MLC LLM or llama.cpp can run 3B models on existing smartphones. You cannot make a judgment without experiencing it.

2. Don’t move everything to local.
Routine tasks should be local, while complex decisions should be cloud-based. This hybrid structure is currently the optimal solution. Thinking in terms of “cloud or local” as a binary choice will lead to failure.

3. Focus on “what was previously impossible” rather than just cost reduction.
Consider the operations that couldn’t use AI because data couldn’t leave, or the sites that couldn’t use AI due to lack of internet. What happens when you introduce local LLMs in those areas? That’s where the true value lies.

This is the Next Phase of “AI Democratization”

When ChatGPT emerged, it was said that “the era of AI accessible to everyone has arrived.” However, the reality was that people were paying monthly subscriptions of several tens of thousands of yen, sending data to the cloud, and relying on internet connectivity.

The era of running LLMs on smartphones overturns that premise. You no longer need monthly fees, cloud services, or internet connectivity to use AI. All you need is a smartphone in your pocket.

The impact of reducing costs from 600,000 yen to 600 yen is significant. However, even more significant is the erasure of the boundary between “companies that can use AI” and “companies that cannot.”

Local factories, farmers in mountainous areas, and constantly traveling salespeople can all carry AI with them. They can stand on the same playing field as large corporations.

This is the true game changer for SMEs.