Conclusion

Let’s get straight to the point. If you are paying more than 50,000 yen a month for cloud AI, you have entered a phase where you should consider in-house operation.

How much are you paying for AI API charges each month? ChatGPT API, Claude API, or Azure OpenAI? As long as you’re spending around 10,000 yen a month, it’s manageable. However, once you start integrating it into your business seriously, costs can quickly rise to 50,000 yen, 100,000 yen, and before you know it, exceed 1 million yen annually. Moreover, all your data is being sent externally.

At this point, the structure is beginning to change. Three simultaneous changes are occurring:

• Hardware: The Apple M5 chip has dramatically improved inference performance.
• Models: Lightweight models with around 10 billion parameters have reached practical levels.
• Tools: Open-source AI agent frameworks have matured.

With these three factors in place, the option of “operating business AI in-house with just one Mac” has become a realistic choice for small and medium enterprises. In this article, we will calculate the cost break-even point and provide specific numbers to illustrate “when it becomes beneficial to operate in-house.”

—

What Changed with the Apple M5 — The Significance of a 6.4x Inference Throughput

The most notable feature of the Apple M5 chip is that it includes dedicated neural accelerators for each core. This differs fundamentally from traditional GPU-based inference designs. The unified memory architecture, which integrates CPU and memory, minimizes the bottleneck of memory transfer even with large models.

Let’s look at some concrete numbers. In tests using the inference runtime “BaseRT,” the inference throughput on the M5 Pro improved by up to 6.4 times compared to the previous M3/M4 generations. What does this mean?

For instance, if a 10 billion parameter model took 6.4 seconds to generate a response per request, the M5 can return it in just 1 second. When used as an internal chatbot or document summarization tool, this speed difference crosses the line between “usable” and “not usable.”

Moreover, what’s crucial is that this can be achieved not with a GPU costing over 1 million yen like the NVIDIA A100, but with a Mac mini or MacBook Pro costing around 300,000 yen. Just power it on and load the model, and it will run. No need for rack-mounted servers or cooling equipment.

—

The Lightweight Model “Domyn-Small” Fills the Last Piece

Hardware alone is not enough. If the model is too heavy to run, it’s meaningless.

Here, the open-weight model “Domyn-Small” from Europe is noteworthy. Despite its lightweight nature with 10 billion parameters, it has the following features:

• 32K token context window (expandable to 128K during inference)
• Multilingual support (including Japanese)
• High accuracy for general tasks such as coding, summarization, classification, and extraction
• Commercially usable open-weight license

Compared to models with 70 billion or 400 billion parameters, there may be scenarios where it falls short in complex inference or creative text generation. However, what small and medium enterprises need are tasks like “summarizing meeting minutes,” “classifying inquiries,” and “drafting standard documents.” A 10 billion parameter model can more than adequately handle these tasks.

Rather, the question should be reversed: “Is a 70 billion model really necessary for that task?”

—

Calculating the Cost Break-Even Point — How Much Does It Take for In-House Operation to Be Beneficial?

Now, let’s get to the main topic. We will estimate the cost break-even point between operating AI on an in-house server (Mac) and continuing to use cloud APIs.

Cost of In-House Operation

| Total | Approximately 34,000 yen/month |

It’s important to make a correction here. The original draft included “AI engineer salary of 300,000 yen per month,” which assumes hiring a full-time engineer. The reality for small and medium enterprises is different. Initial setup is outsourced, and operation is managed with a few hours of maintenance per month. Spot requests for model replacements or fine-tuning are made only when necessary. Under this assumption, the cost can be kept to around 34,000 yen per month.

Cost of Using Cloud APIs

Costs can vary significantly depending on usage, but let’s estimate based on a typical use case for small and medium enterprises.

• Internal chatbot: 50 requests per day, average of 2,000 tokens per request (input and output combined)
• Monthly request count: approximately 1,500 requests
• Monthly token count: approximately 3 million tokens

Calculating with a GPT-4o class API, assuming input costs $2.5 per million tokens and output costs $10 per million tokens (estimated as of June 2025), the monthly cost would be around 3,000 to 5,000 yen. This alone is inexpensive.

However, it doesn’t work that simply in practice.

• Agent-type processing (chaining multiple API calls) can inflate token consumption per task by 10 to 50 times.
• Document processing (contract checks, manual searches, etc.) can consume tens of thousands of tokens per request.
• As the number of users increases, costs rise linearly.

In reality, when integrated into business operations, many companies find themselves in the range of 30,000 to 100,000 yen per month. Furthermore, if combined with RAG (Retrieval-Augmented Generation), hosting costs for vector databases will also be added.

Break-Even Point

The monthly cost of in-house operation is approximately 34,000 yen. In other words —

> Once cloud API charges exceed 35,000 yen per month, in-house operation becomes cheaper.

Moreover, this break-even point tilts in favor of in-house operation as “usage increases.” Cloud services are charged on a pay-per-use basis, so the more you use, the higher the costs. In-house operation has fixed costs, so the more you utilize it, the lower the cost per request becomes.

When employees start using it more because “it’s convenient,” cloud costs will balloon in the form of invoices. With in-house operation, you can say, “Use it as much as you want.” This difference is significant.

—

It’s Not Just About Cost. The Value of Keeping Data In-House

When talking to small and medium enterprise owners, just as much as cost — and sometimes even more — they are concerned about “where the data goes.”

Customer information, estimated amounts, internal meeting minutes. Sending these to a cloud API means transmitting data to an external server. Even if the terms of service state “not used for learning purposes,” it is a rational judgment for business owners to feel “I don’t want to send it outside.”

With in-house operation, data remains entirely within the company’s Mac. No internet connection is even required. This means you can eliminate the hassle of reading terms and conditions and the risk of information leakage altogether.

—

“So, what should I do?” — Three Steps

The reasoning is clear. So, what should you do starting tomorrow?

Step 1: Accurately Assess Current API Charges
First, check your current expenditures. How much are you paying each month? Is the usage trend on the rise? If it exceeds 35,000 yen per month, it’s worth moving to the next step.

Step 2: Test on a Small Scale
There’s no need to transition all operations at once. Buy one Mac mini with the M5 chip, load a model like Domyn-Small, and run just one internal operation. It could be summarizing meeting minutes or answering FAQs. The investment is 300,000 yen plus setup costs. Even if it fails, it’s not a crippling amount.

Step 3: Measure Effectiveness and Decide
Run it for a month and compare speed, accuracy, and costs. If it’s on par with cloud APIs, gradually transition operations. If the accuracy is insufficient, you can keep that operation on the cloud. It’s not all or nothing; optimize with a hybrid approach.

—

Summary: The Era of “AI Belonging to Large Corporations” is Over

With a 300,000 yen Mac and a free open-weight model, you can run your own dedicated AI for a running cost of 34,000 yen per month. Data doesn’t leave your premises. Use it as much as you want.

Three years ago, this would have sounded like a joke. A GPU costing 1 million yen, closed models, and pay-per-use APIs — small and medium enterprises had no choice but to pay costs on the same level as large corporations to use AI.

That structure is now beginning to reverse. The price-performance ratio of hardware has dramatically improved, open-weight models have reached practical levels, and the barriers to setup have lowered.

Because they are small and medium enterprises, decision-making is swift. Because they are small and medium enterprises, one Mac can cover all employees. Because they are small and medium enterprises, they can “buy one and try it first.”

The question is simple. “Will you continue to pay API charges every month, or will you have AI in your own hands?”

There’s no need to wait six months to find the answer. One Mac, 300,000 yen, one month. That’s all it takes to verify.