Conclusion

To put it simply, if your monthly costs exceed 50,000 yen, it’s time to consider operating your own model.

AI API prices are crashing. Literally, they are breaking down on a weekly basis.

From the latter half of 2024 to 2025, the prices of major LLM APIs have plummeted at an astonishing rate. The performance level of GPT-4 can now be accessed at less than one-tenth of the cost from a year ago. Google, Anthropic, and OpenAI are engaged in a price war, driving the cost down to just a few dollars per million input tokens.

This is good news. However, there is a problem.

If you continue to use API pay-as-you-go just because it’s cheap, you may find that your monthly costs have ballooned before you realize it.

Customer inquiries, summarizing internal documents, automatically generating meeting minutes, and marketing reports. As the use cases increase, so do the API calls. While the cost per call may be low, the multiplication can be relentless.

So, when should you switch to an “in-house model”? What is the break-even point in terms of monthly costs?

Let’s verify with specific numbers.

—

First, Accurately Understand Current API Prices

Let’s summarize the price ranges of major APIs as of June 2025 (per million input tokens).

It’s cheap. Indeed, it’s cheap. However, one often overlooked point is that “the output token cost is 3 to 5 times that of the input”. Summarization and text generation typically involve heavy output. If you only look at the input price and think it’s “cheap,” you may be shocked when the bill arrives.

—

Assume Realistic Usage for Small and Medium Enterprises

Let’s consider a small to medium-sized enterprise with 30 to 50 employees. Here are three use cases for AI.

Use Case 1: Customer Inquiry Response (Chatbot)

• Monthly inquiries: 800
• Average tokens per inquiry: 500 input + 1,000 output
• Monthly tokens: 400,000 input + 800,000 output

Use Case 2: Summarizing and Searching Internal Documents

• Monthly documents processed: 300
• Average tokens per document: 3,000 input + 1,500 output
• Monthly tokens: 900,000 input + 450,000 output

Use Case 3: Automatically Generating Sales Reports and Meeting Minutes

• Monthly generation: 100
• Average tokens per report: 2,000 input + 3,000 output
• Monthly tokens: 200,000 input + 300,000 output

Total: 1,500,000 input tokens + 1,550,000 output tokens/month

If processed using the GPT-4.1 class API:

• Input cost: 1.5 × $2 = $3
• Output cost: 1.55 × $8 = $12.4
• Total monthly cost: approximately $15.4 (around 2,300 yen)

It’s cheap. Honestly, for this scale, the API is sufficient. There’s no need to consider an in-house model.

So, when does the balance shift?

—

When Usage Increases Tenfold, the Landscape Changes

Even in small and medium enterprises, as AI utilization progresses, usage can easily increase tenfold.

• Expand the chatbot to all product pages → 8,000 inquiries/month
• Make the entire internal knowledge base searchable by AI → 3,000 documents processed/month
• Automatically generate daily and weekly reports for all sales staff → 1,000 reports/month

Total: 15,000,000 input tokens + 15,500,000 output tokens/month

Using the GPT-4.1 class API:

• Input: 15 × $2 = $30
• Output: 15.5 × $8 = $124
• Total monthly cost: approximately $154 (around 23,000 yen)

Still cheap. But here’s the crux of the matter.

If you want to use a higher-performance model (equivalent to Claude 3.5 Opus, costing $60 to $75 per million output tokens):

• Output cost: 15.5 × $75 = $1,162
• Total monthly cost: approximately $1,200 (around 180,000 yen)

Moreover, when combining RAG (Retrieval-Augmented Generation), the input tokens per instance can swell by 5 to 10 times. This is due to packing a large number of reference documents into the context window.

Recalculating with RAG included:

• Input tokens: 150 million/month
• Output tokens: 15.5 million/month
• With GPT-4.1, the monthly cost is approximately $424 (around 63,000 yen)
• With a high-performance model, the monthly cost is approximately $1,462 (around 220,000 yen)

At this point, operating in-house becomes a viable option.

—

Break Down the Cost Structure of In-House Operations

Let’s realistically estimate the operational costs of an in-house model.

Assuming open models like AMD’s fully open Mixture-of-Experts model “Instella-MoE” or the ultra-compact model “Inkling-Small.”

Pattern A: Running a Small Model (7B-8B parameters) on Local GPU

• Required GPU: NVIDIA RTX 4090 (approximately 250,000 yen) or RTX 5070 Ti (approximately 150,000 yen)
• Server Configuration: 1 GPU + Host PC (approximately 100,000 yen)
• Initial Investment: approximately 250,000 to 350,000 yen
• Monthly Running Costs: Electricity approximately 5,000 yen + Maintenance labor (2 hours/month × 3,000 yen/hour = 6,000 yen)
• Total monthly cost: approximately 11,000 yen

Pattern B: Running a Medium Model (including MoE, effective 14B-30B) on Cloud GPU

• Cloud GPU: 1 A100 spot instance (approximately $1.5/hour)
• Operating Time: 8 hours/day × 22 business days = 176 hours/month
• Monthly GPU Cost: approximately $264 (around 40,000 yen)
• Maintenance Labor: 4 hours/month × 3,000 yen/hour = 12,000 yen
• Total monthly cost: approximately 52,000 yen

Pattern C: Running Cloud GPU 24/7

• Monthly GPU Cost: approximately $1,080 (around 162,000 yen)
• Maintenance Labor: 8 hours/month × 3,000 yen/hour = 24,000 yen
• Total monthly cost: approximately 186,000 yen

—

Calculate the Break-Even Point

To summarize:

The break-even point is around when the monthly API cost exceeds approximately 30,000 to 50,000 yen.

However, this is a simple cost comparison, and there are hidden costs associated with in-house operations.

—

Don’t Ignore the “Hidden Costs” of In-House Operations

While the numbers may suggest that in-house operations are advantageous, there are costs that small and medium enterprises often overlook.

1. Setup Labor
Downloading the model, quantizing it, building the inference server, connecting the RAG pipeline. Do you have personnel within the company who can handle this? If outsourced, expect initial costs of 500,000 to 1,000,000 yen.

2. Model Quality Management
With an API, you are automatically updated to the latest model. In-house operations require you to select, evaluate, and replace models yourself. This can be a surprisingly labor-intensive process.

3. Fault Response
The GPU has broken down. The server has crashed. With an API, you can resolve this with a single phone call, but with in-house operations, you have to fix it yourself.

4. Security
Conversely, this is a benefit of in-house operations. You don’t have to send customer data or internal confidential information externally via the API. Depending on the industry, this alone can be a reason to choose in-house operations.

—

So, What Should You Do?

The strategy for small and medium enterprises is simple when considered in stages.

Step 1: Start with API (monthly cost under 50,000 yen)
Low-cost APIs like GPT-4o mini or Gemini 2.5 Flash are sufficient. You can start with zero initial investment. This is the priority to verify whether “AI can work for your business.”

Step 2: If monthly costs exceed 50,000 yen, consider in-house operations
Explore options to run open models (Llama series, Qwen series, Instella-MoE, etc.) on local GPUs. Use tools like CostPerPrompt to run cost simulations based on your usage patterns.

Step 3: If monthly costs exceed 150,000 yen, seriously consider transitioning
At this scale, the cost benefits of in-house operations become clear. With an on-premise setup of 1-2 GPUs, you can likely operate at less than one-third the cost of the API.

Step 4: Assume a Hybrid Operation
There’s no need to do everything in-house. Routine processes (summarization, classification, template generation) can be handled by in-house models, while complex inference tasks can be managed via APIs. This division of labor is the most cost-effective.

—

What’s Truly Important Is Not the Cost

While we’ve discussed costs up to this point, to be honest, what’s truly important is not the cost.

The fact that AI API prices are crashing weekly means that the “cost of using AI” is approaching nearly zero. In other words, the ability to use AI itself is no longer a differentiator.

What becomes a differentiator is “what tasks you changed and how you changed them using AI.”

A company that automates customer responses for 2,300 yen in API costs and saves the labor of two employees is clearly stronger than a company that spends 500,000 yen just to say, “We have implemented AI.”

What will emerge as costs decrease is a world where “the difference in usage” directly translates to a difference in competitiveness.

Before you ponder whether to go in-house or use an API, I encourage you to try running AI for one task first. We live in an era where you can start for just 2,000 yen. The time spent deliberating is likely to be far more costly.