“We Can’t Function Without That Person” Finally Comes to an End

Creating manuals, responding to inquiries, preparing estimates. In small and medium-sized enterprises, there are countless tasks that can only be done by “that person.” The estimation logic that exists only in the president’s mind, the response manuals known only by veteran employees, and workflows that no one can take over once someone retires.

This “personalization” is the biggest risk for small and medium-sized enterprises.

And now, the conditions to fundamentally break this risk are coming together. The inference costs of LLMs (large language models) have literally dropped to below 1/100 over the past two years.

At the beginning of 2023, processing 1 million tokens with a GPT-4 class model cost about 3,000 to 6,000 yen. As of June 2025, the same processing with GPT-4.1 mini costs about 40 yen. With DeepSeek-V3, it’s around 30 yen. What happens when costs drop to 1/100? It means that the discussion of “whether it can be used” is over, and we have entered the phase of “there’s no reason not to use it.”

Why Have Costs Dropped So Much? — Understand Three Technologies in 30 Seconds

There’s no point in diving too deep into technical details. What small and medium-sized business owners need to grasp is “what has become cheaper and why it has become cheaper.” The key points are three.

① 4-Bit Quantization — A Technology to “Lighten” the Model

The brain of an LLM (parameters) consists of a mass of numbers. Traditionally, each number was represented in 32 bits (32 digits of 0s and 1s). This is compressed down to 4 bits. While accuracy slightly decreases, it remains at a practically acceptable level, reducing necessary memory to about 1/8. Even a model with 12 billion parameters sees memory usage reduced by about 35%, eliminating the need for expensive GPUs (according to LoopKV research). In other words, the hardware costs to run it have drastically decreased.

② KV Cache Compression — A Technology to “Avoid Repeating the Same Calculations”

LLMs use a “cache” to remember past contexts every time they generate text. When this cache grows too large, it consumes memory and increases costs. Research such as FlagRot has demonstrated that by intelligently compressing this cache, costs can be cut by up to 41% while maintaining answer accuracy. It has become possible to provide answers of the same quality with fewer resources.

③ On-Policy Distillation — A Technology to “Transfer the Wisdom of Large Models to Smaller Models”

Problems are solved by a massive model (teacher model) like GPT-4, and the answer patterns are learned by a smaller model (student model). Even if the student model is less than 1/10 the size of the teacher, it can achieve accuracy close to the teacher for specific tasks. There is no longer a need to continue using a general-purpose large model, allowing for the possession of a “small and cheap AI” tailored to company operations.

As a result of these three technologies working together, inference costs have dropped not just to 1/100, but in some cases to below 1/200.

So, How Much Will It Cost? — Monthly Cost Estimates for Three Tasks

This is where the abstract discussion ends. Let’s calculate specific numbers for three common tasks in local small and medium-sized enterprises.

1. Manual Creation: From 500,000 Yen to 8,000 Yen per Month

A common task in manufacturing and construction is the creation and updating of operational manuals. Interviewing veteran employees, formatting in Word, and getting approval from superiors… Outsourcing costs about 10,000 yen per item, and even in-house work costs around 8,000 to 10,000 yen per item when converted to labor costs. If 50 items are created in a month, that’s 500,000 yen.

What happens if this is systematized using LLMs?

• Record the veteran employee’s work as video or audio
• Transcribe using speech recognition tools like Whisper (almost free)
• Format into a manual using LLM (about 3,000 to 5,000 tokens per item)
• For GPT-4.1 mini, the inference cost per item is about 0.2 to 0.5 yen

For 50 items, the inference cost is about 10 to 25 yen. Even including speech recognition and storage, it stays under 8,000 yen per month. Even if a human checks the process, it’s less than 1/50 of the traditional cost.

The key point is that “the veteran’s knowledge remains as text data.” Even if that person leaves, the manual remains, and LLM can draft updates. Personalization structurally disappears.

2. Inquiry Response: From 1,000,000 Yen to 15,000 Yen per Month

“Answering the same questions repeatedly over the phone and email” — a common scenario for small and medium-sized enterprises. For a company with 1,000 inquiries a month, if the cost per response, including labor, is about 1,000 yen, that totals 1,000,000 yen. It’s like employing two part-time workers.

If we automate this with an LLM-based chatbot + RAG (internal document search):

• 70-80% of inquiries are routine questions (delivery confirmations, specification checks, procedural inquiries, etc.)
• These can be automatically answered by the LLM. With about 2,000 tokens per inquiry, the inference cost is about 0.08 to 0.15 yen
• For 700 automated responses, the inference cost is about 56 to 105 yen
• Including the operation of the RAG vector DB and hosting, the monthly cost is about 10,000 to 15,000 yen

Only the remaining 300 complex inquiries are handled by humans. Humans can focus on “work that requires judgment.” Moreover, as logs of automated responses accumulate, the accuracy of the FAQ improves with use.

3. Estimate Preparation: From 250,000 Yen to 5,000 Yen per Month

“Only the president can provide estimates” and “if that salesperson leaves, no one knows the estimation logic” — one of the most personalized tasks in small and medium-sized enterprises.

If 20 estimates are prepared a month at a cost of 12,500 yen per estimate (2 hours of preparation at an hourly rate), that totals 250,000 yen.

In the case of systematizing with LLM:

• Organize past estimate data (items, quantities, unit prices, conditions, final amounts) in CSV format
• Teach the LLM the estimation logic, and when new project conditions are input, it automatically generates a draft estimate
• If a small model specialized for company estimates is created using distillation technology, the inference cost drops further to below 1/10
• The inference cost per estimate is about 1 to 3 yen. For 20 estimates, that’s about 20 to 60 yen
• Including formatting tools and hosting, the monthly cost is about 3,000 to 5,000 yen

The final check is done by a human. However, the time required drops to one-tenth when comparing “creating from scratch” to “checking and modifying a draft.” The logic that was once only in the president’s mind becomes a company asset as data.

Cost Comparison Summary

This Is Not Just a “Copy of Big Corporations” — Reasons Why It Works for Small and Medium-Sized Enterprises

What’s important here is that this is not a story aimed at large corporations.

Large corporations already have specialized departments, established manuals, and outsourced call centers. This means the pain of “personalization” is relatively small.

On the other hand, small and medium-sized enterprises are different. If one person leaves a company of 10, it could mean a 10% loss in revenue. Because the damage from personalization is critical, the impact of systematization through LLMs is exponentially larger.

Moreover, small and medium-sized enterprises can make decisions quickly. “Let’s try it from next week” is feasible. While it may take three months for a large corporation to get approval, a small business can start in three days. Speed is the greatest weapon of small and medium-sized enterprises.

So, What Should We Do Starting Tomorrow?

1. First, choose one task that “only that person can do.” It can be manual creation, inquiry response, or estimates.
2. Verbalize the “inputs and outputs” of that task. What goes in and what comes out. If this can be defined, it can be entrusted to the LLM.
3. Start small. Whether it’s ChatGPT or Claude, try it with a subscription of 2,000 yen per month for 10 tasks. Even with 70% accuracy, just having a draft reduces work time by half.
4. If it works well, switch to an API for automation. This is where the benefits of quantization and distillation come into play. For a few thousand yen a month, you can have “another employee” that operates 24/7.

The fact that inference costs have dropped to 1/100 means that the “cost of trying” has also dropped to 1/100. It’s not painful to fail. So, there’s no reason not to do it.

“We can’t function without that person” — the risk of leaving that state unaddressed is far more costly.