Conclusion

Let’s get straight to the point: the era of “AI is expensive” is coming to an end.

Reflection AI has announced its new model, “Beam.” This is a sparse MoE (Mixture of Experts) model with 501 billion parameters, and its inference cost is $0.15 per 1 million tokens of input. Compared to the input cost of GPT-4o (which is $2.50 per 1 million tokens), this is approximately 1/16th the cost.

What’s even more noteworthy is that Beam can produce small models through “on-policy distillation.” The technology for transferring knowledge from large models to smaller ones has reached a practical stage. In other words, we are entering an era where high-performance inference can be run even cheaper and smaller.

What does this mean for small and medium-sized enterprises (SMEs)? We have entered a phase where the question is no longer whether to use AI, but rather, “What will change in our company when the cost of AI disappears?”

What Exactly is Reflection Beam?

Reflection Beam is a model based on the MoE architecture, with a total of 501 billion parameters. MoE operates by activating only the necessary “expert” modules based on the input, rather than using all parameters every time. This significantly reduces the actual computational load relative to the number of parameters.

According to Reflection AI’s published figures, the inference computation cost is 3 to 4 times lower compared to equivalent open models. It is particularly strong in coding tasks and agent-like workloads (autonomous execution over multiple steps).

The biggest point is “on-policy distillation.” Using the inference results from Beam as training data, knowledge is transferred to smaller models like those with 70 billion or 14 billion parameters. Traditional distillation has primarily been “off-policy,” meaning it trains on existing datasets, but on-policy distillation uses the inference processes generated by the model itself. This allows the “thinking process” of inference to be transferred to smaller models, resulting in minimal degradation in accuracy.

In short, the quality produced by large models can be replicated by smaller models. The costs drop dramatically.

Estimating Costs for SMEs in Three Scenarios

So, what would the costs look like when SMEs use AI inference? Let’s consider three scenarios: monthly subscription, in-house operation, and API pay-per-use.

Scenario 1: Monthly Subscription (e.g., ChatGPT Pro)

Currently, many SMEs are using this model. ChatGPT Plus costs $20 per person per month, while Pro costs $200 per person per month. For 10 employees, this amounts to $2,000 to $20,000 per month (approximately 300,000 to 3,000,000 yen).

This monthly subscription model may seem like “unlimited use,” but it cannot be used for automation via API. It is limited to human interaction with a screen. In other words, it does not lead to business automation.

While the monthly subscription is excellent for “trial use,” as long as one remains in this model, AI will remain just a “convenient tool.”

Scenario 2: API Pay-Per-Use

This is the main contender. Here, inference is called via API and integrated into business workflows. This is where the cost structure of Beam class comes into play.

Let’s do some specific calculations:

• A local manufacturing company summarizes and classifies 200 daily reports using AI.
• Assuming 1,000 tokens of input and 500 tokens of output per report.
• With 22 working days in a month, that totals 4,400 reports.

Assuming Beam class costs (input $0.15/1 million tokens, output $0.60/1 million tokens):

• Input: 4,400 × 1,000 = 4.4 million tokens → approximately $0.66
• Output: 4,400 × 500 = 2.2 million tokens → approximately $1.32
• Total monthly cost: approximately $1.98 (about 300 yen)

For GPT-4o API (input $2.50/1 million tokens, output $10.00/1 million tokens):

• Input: approximately $11.00
• Output: approximately $22.00
• Total monthly cost: approximately $33.00 (about 5,000 yen)

Both options are inexpensive. However, it’s important to note that this is for “one business function.” What happens when you expand beyond daily reports to include creating estimates, responding to inquiries, inventory analysis, and meeting minutes? Even if token consumption increases tenfold, with Beam class, it would still be around 3,000 yen per month. For GPT-4o, it would be around 50,000 yen.

Tasks that previously cost 100,000 to 300,000 yen per month when outsourced can now be automated for just a few thousand yen per month. This structural change is essential.

Scenario 3: In-House Operation (On-Premises/Private Cloud)

This scenario involves operating Beam’s distillation model (70B class) in-house. For industries that cannot expose customer data or internal secrets (such as healthcare, finance, or professional services), this becomes a realistic option.

A rough estimate of the necessary infrastructure:

• NVIDIA A100 80GB × 2: approximately $3,000 to $4,000 per month in the cloud (AWS/GCP)
• If purchased on-premises, about 2 million yen per card × 2 = approximately 4 million yen (depreciated over 3 years, about 110,000 yen per month)
• Personnel costs for operation and maintenance: 100,000 to 200,000 yen per month

This totals around 200,000 to 300,000 yen per month. This may feel “expensive.” However, if this model can automatically handle 5 to 10 business functions 24/7, it can replace over 1 million yen in personnel costs per month. The return on investment can be recouped in less than six months.

However, to be honest, it is risky for SMEs to jump straight into in-house operation. A more realistic approach would be to first validate “which business functions AI can effectively enhance” through API pay-per-use, and then consider in-house operation once token consumption and effects are clear.

What Really Changes is Not “Cost” but “Structure”

While we have discussed costs up to this point, the essence is not merely cost reduction.

When inference costs drop to one-hundredth, the criteria for deciding whether a task is done by a human or AI changes.

Previously, the calculation was, “If I implement AI in this task, will it save 100,000 yen a month?” Only tasks that showed a favorable return on investment were targeted for AI implementation.

However, if inference costs drop to a few hundred yen per month, calculating ROI becomes unnecessary. It becomes rational to “just run everything through AI.” Daily reports, emails, estimates, inquiries—everything.

This structure is more advantageous for SMEs than for large corporations. Why?

Large corporations have to go through approval processes, security reviews, vendor selections, and proof of concept periods. It takes six months to AI-enable a single business function.

In contrast, SMEs can start moving as soon as the CEO says, “Let’s do it.” They can connect 10 business functions to the API simultaneously and keep only those that show effectiveness. The speed of decision-making multiplies with the commoditization of technology. This is the structure of reversal.

What Mass Production of Distillation Models Means

I would like to delve deeper into the topic of distillation.

The ability to mass-produce small models from large models like Beam means that we can create “business-specific small AIs” at a low cost.

For example, a 14B model specialized for construction estimates, a 7B model specialized for summarizing care records, or a 3B model specialized for agricultural shipment judgments.

Creating business-specific models used to cost hundreds of thousands to millions of yen in data collection, annotation, and training. With distillation, you can process business data with a large model and use its output as training data for a small model. Costs could drop to the tens of thousands of yen range.

Moreover, small models can run on smartphones and edge devices. Inference can occur on-site using tablets without internet connectivity. AI can be used even in remote construction sites or on fishing boats.

The premise that “AI cannot be used without a connection to the cloud” is collapsing. This is a critically important change for local SMEs.

So, What Should We Do?

Just three things.

1. First, decide on one business function to automate via API
It can be summarizing daily reports or automatically classifying inquiries—anything works. You can try it for a few hundred yen. It won’t hurt if it fails.

2. List out “tasks that don’t need to be done by humans”
Assuming inference costs approach zero, take stock of internal operations. Ask, “What would happen if I threw this to AI?” for everything.

3. Keep an eye on the trends in distillation models
Not just Beam, but also Llama, Qwen, Gemma, and other open models are rapidly maturing in the distillation ecosystem. Having a small model specialized for your business operations will determine your competitiveness in 1 to 2 years.

The world where inference costs drop to one-hundredth is already here. The only question is, “When will you start?” Will it be next year or next week? That difference will determine the state of your company three years from now.