Conclusion First

“Throwing everything at the same model” is the most expensive option.

Small and medium-sized enterprises (SMEs) paying 100,000 yen a month in API costs are likely wasting about 30,000 yen of that.

The reason is simple: whether it’s rough summaries for internal chats or carefully crafted emails for customers, everything is being sent to the same GPT-4o model. This is akin to calling a taxi just to go to the nearby convenience store.

Ramp has announced that its “AI Model Router” has reduced its own LLM operational costs by 30%, providing a straightforward solution to this issue. It automatically allocates tasks to high-performance models or lightweight models based on their complexity. That’s all there is to it. Yet, just by doing “that,” you can save 30,000 yen a month. That’s 360,000 yen a year. For SMEs, this amount can be redirected to part of their labor costs.

What is the AI Model Router?—Explained in Three Lines

1. The user sends an API request.
2. The router determines the complexity of the task.
3. Simple tasks are allocated to GPT-4o mini or Claude 3.5 Haiku, while only complex tasks are sent to GPT-4o or Claude 3.5 Sonnet.

That’s it. There’s nothing technically challenging about it.

The important aspect is the structure of “why this reduces costs.” Looking at OpenAI’s pricing table makes it clear. The input token cost for GPT-4o is about $2.50 per million tokens, while GPT-4o mini is around $0.15. There’s a difference of about 17 times. The difference in output tokens is similarly stark, with $10 versus $0.60.

In other words, if 70% of all requests are tasks that “mini can handle,” the cost for that 70% drops to less than one-tenth. Rather than a 30% reduction overall, a reduction of over 50% becomes realistic depending on usage.

Viewing Common Scenarios for SMEs in Numbers

Let’s break down the typical monthly API costs for a regional SME that is actively utilizing AI.

Which of these truly requires the performance of GPT-4o?

Only the customer proposals and data analysis & reporting need it. Internal FAQs, email drafts, and meeting minutes can achieve sufficient quality with GPT-4o mini or Claude 3.5 Haiku.

Here’s how the cost estimation looks when tasks are allocated via the router.

From 100,000 yen to 47,000 yen. A 53% reduction. A difference of about 640,000 yen annually.

Ramp’s announced 30% reduction is a conservative figure. In cases like SMEs with simple use cases that are currently using “all high-end models,” the potential for cost reduction is even greater.

Design Patterns for Building Your Own

While Ramp’s router is an internal tool for their company, a similar system can be built by SMEs as well. Here are three patterns to consider.

Pattern 1: Rule-Based Allocation (The Simplest)

Set up a single proxy server to receive API requests. Based on the type of request (endpoints and parameters), switch models according to pre-defined rules.

• Internal FAQ → GPT-4o mini
• Customer-facing content generation → GPT-4o
• Summarization tasks → GPT-4o mini

The construction cost is minimal; if you have an engineer who can write Python, it would take 1-2 days. Even if outsourced, it would only cost around 100,000 to 200,000 yen. If you save 50,000 yen a month, you’d recoup the costs in 2-4 months.

Pattern 2: Utilizing LiteLLM or OpenRouter

Using open-source LiteLLM or API aggregators like OpenRouter makes switching between multiple models even easier. LiteLLM provides an OpenAI-compatible API while routing to over 100 models in the backend.

The advantage is that you can implement it with almost no changes to your existing code. You can change the model allocation rules simply by modifying the configuration file.

Pattern 3: Combining with Local Inference (Advanced)

Set up a local inference server using Ollama or vLLM, processing simple tasks locally. Only complex tasks that cannot be handled locally fall back to the cloud API.

This hybrid structure can compress API costs even further. However, it requires an initial investment in GPU servers (around 300,000 to 500,000 yen), making it suitable for companies with monthly API costs exceeding 200,000 yen. For those below 100,000 yen, Patterns 1 or 2 would suffice.

Setting a Budget Limit—Preventing “Accidental Overuse”

Another thing SMEs should do starting today is to set a budget limit for API usage.

OpenAI’s API has a feature called “Usage limits.” By setting a monthly cap, the API will stop once that limit is exceeded. The Claude API has a similar feature.

Surprisingly, many SMEs do not have this set up. I’ve heard numerous stories of employees experimenting with the API, only to pale when they see the bill at the end of the month.

Setting this up takes just 5 minutes. Log into the dashboard and input the amount. If you haven’t done this yet, I urge you to do it before finishing this article.

Large Corporations with GPU Utilization Below 50%—Don’t Let SMEs Follow This Path

According to research from Stanford HAI and others, it’s not uncommon for large corporations to have GPU clusters with utilization rates below 50%. They spend hundreds of millions of yen on GPUs, leaving them idle for more than half the time. The mindset of “just securing high specs” in large companies leads to waste.

SMEs do not need to mimic this mindset at all. In fact, there are things that only SMEs can do.

• Because their use cases are limited, they can more easily select the optimal model.
• Because decision-making is swift, they can quickly switch to new, cheaper models as they become available.
• Because one person can grasp the overall usage across the company, it’s easier to identify waste.

The cost structures that large corporations cannot optimize due to organizational complexity can be changed by SMEs with a simple directive from the CEO. This is a clear advantage.

So, What Should You Do?

Let’s narrow it down to three actions you can take today.

1. Break Down Your Company’s API Usage by Purpose (30 minutes)
Understand what you’re spending and on what. By opening OpenAI’s Usage dashboard, you can see costs broken down by day and model.

2. Identify Tasks That Can Be Handled by “mini” and Switch Models (1 hour)
For internal summaries, FAQs, and draft generation, first switch to GPT-4o mini and check the quality. If there are no issues based on your experience, continue using it.

3. Set a Monthly Budget Limit (5 minutes)
Input the limit amount in the API dashboard. This alone will eliminate “unexpected bills.”

With these three steps, your monthly API costs will likely drop by 30-50%. No special tools, outsourcing, or advanced technical skills are required.

The cost of AI is not a matter of “high or low.” It’s about whether you’re using it appropriately. Companies throwing everything at GPT-4o are like those using taxis for every trip. They could just walk to the nearby convenience store.

Saving 30,000 yen a month is not a trivial amount for SMEs. With that 30,000 yen, you could conduct another AI experiment.